This document summarizes the results of an eye tracking study commissioned by ICOMP to analyze attention and selection behavior on Google search result pages. The study found that: 1) Google's own product search results and map/image panes received the majority of visual attention from users and were clicked on most frequently. 2) Alternative search sites logos received little visual attention and few clicks. 3) Organic links received less visual attention than Google elements above them and were often not noticed due to their positioning lower on the page. 4) Google ads received very little visual attention or clicks across all search result pages tested.

1. Attention
and
selection
behavior
on
"universal
search" result pages based
on
proposed
Google
commitments of Oct. 21, 2013
Report about an eye tracking pilot study
commissioned by ICOMP Initiative for a
Competitive Online Marketplace
Introduction
2
Short report
4
Setup of the study
5
Detailed results of the study
14
References
23
Institute of Communication and Media Research
GERMAN SPORTS UNIVERSITY COLOGNE
- November 21st, 2013 -

2. 1. Introduction
The pilot study as described in the following
illustrates an empirical approach regarding the
distribution of attention and the selection
behavior on Google "Universal Search" result
pages.
The ISO Norm line 9241-11states as the three
criteria for usability: effectiveness, efficiency
and satisfaction. One tries to adjust the
interface design to these criteria for example by
user tests, interviews, heuristic evaluation by
experts
or
modeling
the
expected
user
behavior.
In markets with a large variety of offers and little
possibility of differentiation, providers can gain
a decisive competitive advantage by user
oriented interfaces. A precondition of this is that
relevant
information
can
be
obtained
for
entrepreneurial decisions to this regard.
2

3. Commissioned
by
the
Initiative
for
a
Competitive Online Marketplace ICOMP, the
Institute
Research
of
Communication
(IKM)
at
the
and
Media
German
Sports
University Cologne (DSHS) developed and
realized a suitable procedure. In the following
• the main findings are briefly presented
• the methods are discussed
• individual results are briefly explained
A pilot study could be conducted within the time
and budget constraints, which already presents
clear results.
3

4. 2. Short report
(1)
Google placed "Sponsored" Googleown page elements grab a predominant amount of the total visual
attention.
(2)
The "alternative search sites" even
with small logos do not evoke enough
visual attention to stimulate the users
to click on them.
(3)
Map services can not compete
against Google because the placed
Google Map pane and Google Images
thumbnails get visual attention of
users more, earlier and longer than all
other page elements.
(4)
Google Ads presented as lateral
skyscraper-shaped link lists are
vastly irrelevant for visual attention
as well as for mouse clicking
behavior.
(5)
The visual attention for organic links
on the SERPs is negligible compared
to those Google elements placed
above them, enhanced with pictures.
With browser windows not opened
wide, organic links may go unseen.
4

5. 3. Setup of the study
The study presented here represents a user
oriented
study
using
a
special
kind
of
observation.
By interviewing, general usage habits regarding
the users' preferred web browsers, a specific
search engine or a specific online service can
be learned. Data on questionnaires cannot
represent the behavior retrospectively because
this is affected, for example, by memory effects
(Felix, Hink & Minor, 2001; Möller & Schierl,
2012). Therefore an eye tracker was used
additionally. This was a high resolution device
of the company Tobii, based on the cornea
reflex method (Duchowski, 2007). The use of
this elaborate method is necessary in order to
catch spontaneous user behavior that is not
adulterated by willful processes (Byrne et al.,
1999;
Graham & Jefferey, 2011;
Ferreira et al.,
5

6. 2011). In respect to methodology Just and
Carpenter (1980) assumed that the users'
visual attention is focused on the object that is
also the object of the cognitive processing (eyemind hypothesis). It is further assumed that
the time of fixation corresponds to the time of
cognitive
processing
(immediacy
assumption).
35 persons were recruited as test subjects, 20
men and 15 women. The specification was to
measure eye movements on search engine
result pages (SERPs) given as examples in the
October 21, 2013 commitment proposal by
Google, Annex V, of the Case COMP/C3/39.740 – Foundem and others, and an
additional SERP depicting Google Maps. For
this, the following three SERPs were used as
target pictures.
6

7. Fig.1: Target picture "iPod shopping" (from
Annex V, P. 35)
7

8. Fig.2: Target picture "map London" (screenshot
from actual Google web search engine)
8

9. Fig.3: Target picture "flight search" (from Annex
V, p.39)
In order to have the subjects behave as they
would in normal, everyday Internet use, the
purpose of the test was not revealed to them
and to preserve their attention for the stimuli
9

10. used during the entire measuring cycle, a
special test setting was created.
The subjects sat about 2 meters in front of a 46inch plasma screen monitor that showed
various images. The eye tracker stood about 70
centimeters (= 2,297 ft) in front of the subjects
and therefore was outside the regular sight field
of the participants. All instructions, stimuli and
questions were only shown on the screen to
avoid
any
distortions
caused
by
the
experimenter. In order to get the subjects used
to the situation and gain their attention they
were informed that they were participating in a
perception experiment and were presented
several images and questions unrelated to the
SERPs to be examined; They were presented
with a Stroop test of word recognition and
thereafter with a geometrical-optical illusion
image. Then the subjects were asked to look at
an advertisement for eight seconds. In the
advertisement, a spokesperson with a visible
10

11. physical handicap was promoting a fictional
automotive brand. Besides the fictional brand,
we
questioned
conspicuous
the
subjects
issues
about
concerning
any
the
spokesperson. Then the subjects got written
instructions to click on a link they desired in an
on-line advertisement. Then they were asked
about any sponsors visible on this site. Next the
subjects were shown a picture of a table tennis
athlete in a tuxedo and asked to assign a name
to him from a list of choices. Then the
participants were shown a picture of the same
athlete performing his sport and were asked for
a name choice. After this subjects were asked if
they are iPod users, if they usually buy those
electronic
devices
in
specialised
shops,
discount shops, or online shops. After this
subjects were instructed to assume intention to
buy an iPod nano and select one of six search
queries
by
mouse
click.
The
instruction
continued that after their choice a Google
11

12. search result page will be visible and that
subjects should click on it to proceed (cf. fig. 1,
taken from Annex V, p. 35) buying an iPod
nano. Next instruction was to memorise as
many details of an upcoming iPod ad. The ad
was visible for six seconds. Then subjects had
to answer how many apps were visible on the
display of iPod that was presented in the ad.
After that subjects were asked to use a SERP
to choose a map to find Apple Stores in London
(cf. fig.2, screenshot from actual Google web
search engine). Then they were asked if they
ever
booked
a
flight
on
the
web.
The
information was given that web booking is very
usual in the US. The next SERP showed
different flight offers (cf. fig.2, taken from Annex
V, p. 39). Subjects should choose one of the
offerings, again by clicking as usual. After this
subjects were thanked for their participation and
dismissed.
12

13. The
eye
movements
and
fixations
were
recorded at a rate of 120 Hertz during the entire
measuring section. Video and audio of all
subjects was recorded with a webcam in
parallel
and
integrated
with
the
stimuli
presented, the mouse clicks and the tracking
data into an overall view.
For each target picture the following parameters
were computed from the raw data:
1) Fixation count
2) Absolute duration of the fixations
3) Time to first fixation.
A fixation occurs when an eye movement rests
for at least 200 milliseconds on an area of 50
pixels.
The results arrived at are described in detail in
the following chapter.
13

14. 4. Detailed results of the study.
As the first analytical step, the left mouse clicks
of the subjects were marked in the three target
pictures as symbolized (
) mouse-left-click.
At the same time these data were to be
complemented
by
the
data
of
the
eye
movement. Therefore corresponding heat maps
were generated. In these heat maps, a color
code indicates the diverse intensities of the
visual attention triggered by the SERPs. Analog
to a traffic light, the color red represents the
maximal viewing time and the color green the
minimal duration. In all areas without color
assignment there was no fixation. For optimal
comparability, each heat map is shown on a
single page followed by a table containing the
corresponding data. Results will be discussed
below.
14

15. Fig.4: Mouse clicks and total visual attention on
target picture "iPod shopping"
15

16. Table 1: Overview of the recorded parameters
on target picture "iPod shopping"
Area
of Fixation
Time to Fixation Mouse
interest
duration First
(AOI)
(FD)
Counts
Fixation (FCʼ)
Clicks
(MC)
(TFF)
ad on top 0.54
1.08
192
0
1.10
0.27
715
18
“alternative 0.67
3.11
311
1
0.35
459
13
of SERP
Googles
own
product
search
results
sites”
organic
1.60
blue links
16

17. ig.5: Mouse clicks and total visual attention on
the target picture "map London"
17

18. Table 2: Overview of the recorded parameters
on target picture "map London"
AOI
FD
TFF
FC
MC
Google map
0.77 0.46 286 15
“Google images”
0.98 0.62 348 12
organic blue links 1.39 2.87 322 6
18

19. Fig.6: Mouse clicks and total visual attention on
the target picture "flight search"
19

20. Table 3: Overview of the recorded parameters
on target picture "flight search"
AOI
FD
TFF
FC
MC
ad on top of SERP
0.85 1.50 143 2
Googles own product 2.03 0.10 556 15
search results
“alternative sites”
0.91 5.54 110 4
organic blue links
1.16 3.35 196 11
Skyscraper-shaped ads 1.66 3.93 174 3
20

21. The SERP for the search term "iPod" reveals
that thumbnail product pictures guide the
spontaneous visual attention of users to the
"Google Shopping Results". For this area the
time to first fixation is least: 0.27 seconds. This
concurs with the amount of mouse clicks: 56%
of the participants clicked into this area with
Google's own product search. The average
amount of visible attention (the total fixation
duration) is 1.1 seconds. The maximum total
duration of fixations on the organic links is 1.6
seconds. 41% clicked one of the organic blue
links, mostly the first (Apple.com). One has to
keep in mind that the organic links were fixated
for the first time only after 3.5 seconds in
average! This is a long period, even for highly
involved users. To adhere to Google's example,
and
to
save
time,
the
original
mockup
screenshot from their commitments paper was
used, which contained Apple.com as first
ranked organic blue link. Further testing on
21

22. SERP examples without the very well known
brand Apple leading the organic blue links,
might show less attention and clicks to this
position and even more in the product images.
The related ad at the top of the SERP had no
clicks and the "alternative search sites" were
scanned visually even less and only clicked
once. Both show a very small time in fixation
duration of round about half a second. One may
conclude that these areas are evoking no
interest from recipients.
On the target SERP "map London" 46% of the
participants
clicked
on
the
Google
map
whereas 36% clicked on the area of "Google
Images"
which
refers
to
another
Google
service. Also the total duration of fixations for
both areas is comparable (0.77 seconds for the
Google map, 0.98 seconds for the "Google
images"). But the Google map gets visual
attention of the user faster (0.49 seconds)
22

23. compared
seconds).
to
the
"Google
Number
mylondonmap.com
one
is
a
Image"
(0.62
organic
link
site
using
the
syndicated Google Maps API and received two
clicks. The famous, official Transport for
London maps received no click. Further down
the SERP organic links again referring to
Google Maps received four clicks together.
The "flight search" SERP in fig. 3 is the
visually most complex stimulus in the study
design. Similar to the SERP "iPod shopping"
only few mouse clicks are on the ad on the top
of the SERP (two clicks = 6%). Most clicks are
on the Google Flight Search "sponsored links"
area (43%), followed by the "organic links" area
(31%). The "alternative search sites" had little
visual attention with correspondingly only four
clicks (11%) and the ads on the right side just
three clicks (9%). Next to the amount of mouse
clicks, the total visual attention (2.03 seconds)
as well as the very small time to first fixation
23

24. (0.10 seconds) indicates that the Google Flight
Search area is the most attractive one of this
SERP. It is remarkable that even the organic
links at the bottom of the Page get more visual
attention (1.16 seconds) and faster (3.35
seconds) than the "alternative search sites"
(duration 0.91 seconds; time to first fixation =
5.54 seconds).
Finally all target pictures suggest that the
implemented "Sponsored (i)" labels do not
reach the beholder’s visual attention. This
finding is supported by the fact that those
labels were not clicked.
24

25. 5. References
Byrne, M.D., Anderson, J.R., Douglass, S.
and Matessa, M. (1999). Eye tracking the
visual search of click-down menus. In
Human Factors in Computing Systems:
CHIʼ99 Conference Proceedings, ACM
Press, New York, pp. 402-409.
Duchowski,
A.(2007).
Methodology.
Theory
Springer, London.
Eye
and
Tracking
Practice.
Felix, R., Hink, W. and Minor, M. (2001).
Hereʼs looking at you kid: Eye-tracking
applications in consumer information
processing research. In Proceedings of the
Annual Meeting of the Society for
Marketing Advances, pp. 157-161.
Ferreira, P., Rita, P., Morais, P., Oliveira, J.,
Gamito, P., Santos, N., Soares, F. and
Sottomayor, C. (2011). Grabbing attention
while reading website pages: the influence
of verbal emotional cues in advertising.
Journal of Eyetracking, Visual Cognition
and Emotion, 1, 1, pp. 64-68.
Graham, D.J. and Jefferey, R.W. (2011).
Location, Location, Location: Eye-Tracking
Evidence that Consumers Preferentially
View Prominently Positioned Nutrition
Information. Journal of the American Diet
Association. 111, 11, 1704-1711. doi:
25

26. 10.1016/j.jada.2011.08.005
Just, M.A. and Carpenter, P.A. (1980). A
theory of reading: from eye fixation to
comprehension. Psychological Review, 87,
4, pp. 329–354.
Möller, C. and Schierl, T. (2012). Attention
and selection behavior on "universal
search"
result
pages.
Journal
of
Eyetracking, Visual Cognition and Emotion,
2, 1, pp. 1-10.
26

27. Institute of Communication and
Research
German Sport University Cologne
Am Sportpark Müngersdorf 6
50933 Cologne
Germany
Media
Contact:
Dr. Carsten Möller
c.moeller@dshs-koeln.de
+49(0)221 4982 6243
www.ikm-dshs.de
27