Bericht icomp vol4


Published on

Published in: Technology, News & Politics
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Bericht icomp vol4

  1. 1. Attention and selection behavior on "universal search" result pages based on proposed Google commitments of Oct. 21, 2013 Report about an eye tracking pilot study commissioned by ICOMP Initiative for a Competitive Online Marketplace Introduction 2 Short report 4 Setup of the study 5 Detailed results of the study 14 References 23 Institute of Communication and Media Research GERMAN SPORTS UNIVERSITY COLOGNE - November 21st, 2013 -
  2. 2. 1. Introduction The pilot study as described in the following illustrates an empirical approach regarding the distribution of attention and the selection behavior on Google "Universal Search" result pages. The ISO Norm line 9241-11states as the three criteria for usability: effectiveness, efficiency and satisfaction. One tries to adjust the interface design to these criteria for example by user tests, interviews, heuristic evaluation by experts or modeling the expected user behavior. In markets with a large variety of offers and little possibility of differentiation, providers can gain a decisive competitive advantage by user oriented interfaces. A precondition of this is that relevant information can be obtained for entrepreneurial decisions to this regard.   2  
  3. 3. Commissioned by the Initiative for a Competitive Online Marketplace ICOMP, the Institute Research of Communication (IKM) at the and Media German Sports University Cologne (DSHS) developed and realized a suitable procedure. In the following • the main findings are briefly presented • the methods are discussed • individual results are briefly explained A pilot study could be conducted within the time and budget constraints, which already presents clear results.   3  
  4. 4. 2. Short report (1) Google placed "Sponsored" Googleown page elements grab a predominant amount of the total visual attention. (2) The "alternative search sites" even with small logos do not evoke enough visual attention to stimulate the users to click on them. (3) Map services can not compete against Google because the placed Google Map pane and Google Images thumbnails get visual attention of users more, earlier and longer than all other page elements. (4) Google Ads presented as lateral skyscraper-shaped link lists are vastly irrelevant for visual attention as well as for mouse clicking behavior. (5) The visual attention for organic links on the SERPs is negligible compared to those Google elements placed above them, enhanced with pictures. With browser windows not opened wide, organic links may go unseen.   4  
  5. 5. 3. Setup of the study The study presented here represents a user oriented study using a special kind of observation. By interviewing, general usage habits regarding the users' preferred web browsers, a specific search engine or a specific online service can be learned. Data on questionnaires cannot represent the behavior retrospectively because this is affected, for example, by memory effects (Felix, Hink & Minor, 2001; Möller & Schierl, 2012). Therefore an eye tracker was used additionally. This was a high resolution device of the company Tobii, based on the cornea reflex method (Duchowski, 2007). The use of this elaborate method is necessary in order to catch spontaneous user behavior that is not adulterated by willful processes (Byrne et al., 1999;   Graham & Jefferey, 2011;   Ferreira et al.,   5  
  6. 6. 2011). In respect to methodology Just and Carpenter (1980) assumed that the users' visual attention is focused on the object that is also the object of the cognitive processing (eyemind hypothesis). It is further assumed that the time of fixation corresponds to the time of cognitive processing (immediacy assumption). 35 persons were recruited as test subjects, 20 men and 15 women. The specification was to measure eye movements on search engine result pages (SERPs) given as examples in the October 21, 2013 commitment proposal by Google, Annex V, of the Case COMP/C3/39.740 – Foundem and others, and an additional SERP depicting Google Maps. For this, the following three SERPs were used as target pictures.   6  
  7. 7. Fig.1: Target picture "iPod shopping" (from Annex V, P. 35)   7  
  8. 8. Fig.2: Target picture "map London" (screenshot from actual Google web search engine)   8  
  9. 9. Fig.3: Target picture "flight search" (from Annex V, p.39) In order to have the subjects behave as they would in normal, everyday Internet use, the purpose of the test was not revealed to them and to preserve their attention for the stimuli   9  
  10. 10. used during the entire measuring cycle, a special test setting was created. The subjects sat about 2 meters in front of a 46inch plasma screen monitor that showed various images. The eye tracker stood about 70 centimeters (= 2,297 ft) in front of the subjects and therefore was outside the regular sight field of the participants. All instructions, stimuli and questions were only shown on the screen to avoid any distortions caused by the experimenter. In order to get the subjects used to the situation and gain their attention they were informed that they were participating in a perception experiment and were presented several images and questions unrelated to the SERPs to be examined; They were presented with a Stroop test of word recognition and thereafter with a geometrical-optical illusion image. Then the subjects were asked to look at an advertisement for eight seconds. In the advertisement, a spokesperson with a visible   10  
  11. 11. physical handicap was promoting a fictional automotive brand. Besides the fictional brand, we questioned conspicuous the subjects issues about concerning any the spokesperson. Then the subjects got written instructions to click on a link they desired in an on-line advertisement. Then they were asked about any sponsors visible on this site. Next the subjects were shown a picture of a table tennis athlete in a tuxedo and asked to assign a name to him from a list of choices. Then the participants were shown a picture of the same athlete performing his sport and were asked for a name choice. After this subjects were asked if they are iPod users, if they usually buy those electronic devices in specialised shops, discount shops, or online shops. After this subjects were instructed to assume intention to buy an iPod nano and select one of six search queries by mouse click. The instruction continued that after their choice a Google   11  
  12. 12. search result page will be visible and that subjects should click on it to proceed (cf. fig. 1, taken from Annex V, p. 35) buying an iPod nano. Next instruction was to memorise as many details of an upcoming iPod ad. The ad was visible for six seconds. Then subjects had to answer how many apps were visible on the display of iPod that was presented in the ad. After that subjects were asked to use a SERP to choose a map to find Apple Stores in London (cf. fig.2, screenshot from actual Google web search engine). Then they were asked if they ever booked a flight on the web. The information was given that web booking is very usual in the US. The next SERP showed different flight offers (cf. fig.2, taken from Annex V, p. 39). Subjects should choose one of the offerings, again by clicking as usual. After this subjects were thanked for their participation and dismissed.   12  
  13. 13. The eye movements and fixations were recorded at a rate of 120 Hertz during the entire measuring section. Video and audio of all subjects was recorded with a webcam in parallel and integrated with the stimuli presented, the mouse clicks and the tracking data into an overall view. For each target picture the following parameters were computed from the raw data: 1) Fixation count 2) Absolute duration of the fixations 3) Time to first fixation. A fixation occurs when an eye movement rests for at least 200 milliseconds on an area of 50 pixels. The results arrived at are described in detail in the following chapter.   13  
  14. 14. 4. Detailed results of the study. As the first analytical step, the left mouse clicks of the subjects were marked in the three target pictures as symbolized ( ) mouse-left-click. At the same time these data were to be complemented by the data of the eye movement. Therefore corresponding heat maps were generated. In these heat maps, a color code indicates the diverse intensities of the visual attention triggered by the SERPs. Analog to a traffic light, the color red represents the maximal viewing time and the color green the minimal duration. In all areas without color assignment there was no fixation. For optimal comparability, each heat map is shown on a single page followed by a table containing the corresponding data. Results will be discussed below.   14  
  15. 15. Fig.4: Mouse clicks and total visual attention on target picture "iPod shopping"   15  
  16. 16. Table 1: Overview of the recorded parameters on target picture "iPod shopping" Area of Fixation Time to Fixation Mouse interest duration First (AOI) (FD) Counts Fixation (FCʼ) Clicks (MC) (TFF) ad on top 0.54 1.08 192 0 1.10 0.27 715 18 “alternative 0.67 3.11 311 1 0.35 459 13 of SERP Googles own product search results sites” organic 1.60 blue links   16  
  17. 17. ig.5: Mouse clicks and total visual attention on the target picture "map London"   17  
  18. 18. Table 2: Overview of the recorded parameters on target picture "map London" AOI FD TFF FC MC Google map 0.77 0.46 286 15 “Google images” 0.98 0.62 348 12 organic blue links 1.39 2.87 322 6   18  
  19. 19. Fig.6: Mouse clicks and total visual attention on the target picture "flight search"     19  
  20. 20. Table 3: Overview of the recorded parameters on target picture "flight search" AOI FD TFF FC MC ad on top of SERP 0.85 1.50 143 2 Googles own product 2.03 0.10 556 15 search results “alternative sites” 0.91 5.54 110 4 organic blue links 1.16 3.35 196 11 Skyscraper-shaped ads 1.66 3.93 174 3   20  
  21. 21. The SERP for the search term "iPod" reveals that thumbnail product pictures guide the spontaneous visual attention of users to the "Google Shopping Results". For this area the time to first fixation is least: 0.27 seconds. This concurs with the amount of mouse clicks: 56% of the participants clicked into this area with Google's own product search. The average amount of visible attention (the total fixation duration) is 1.1 seconds. The maximum total duration of fixations on the organic links is 1.6 seconds. 41% clicked one of the organic blue links, mostly the first ( One has to keep in mind that the organic links were fixated for the first time only after 3.5 seconds in average! This is a long period, even for highly involved users. To adhere to Google's example, and to save time, the original mockup screenshot from their commitments paper was used, which contained as first ranked organic blue link. Further testing on   21  
  22. 22. SERP examples without the very well known brand Apple leading the organic blue links, might show less attention and clicks to this position and even more in the product images. The related ad at the top of the SERP had no clicks and the "alternative search sites" were scanned visually even less and only clicked once. Both show a very small time in fixation duration of round about half a second. One may conclude that these areas are evoking no interest from recipients. On the target SERP "map London" 46% of the participants clicked on the Google map whereas 36% clicked on the area of "Google Images" which refers to another Google service. Also the total duration of fixations for both areas is comparable (0.77 seconds for the Google map, 0.98 seconds for the "Google images"). But the Google map gets visual attention of the user faster (0.49 seconds)   22  
  23. 23. compared seconds). to the "Google Number one is a Image" (0.62 organic link site using the syndicated Google Maps API and received two clicks. The famous, official Transport for London maps received no click. Further down the SERP organic links again referring to Google Maps received four clicks together. The "flight search" SERP in fig. 3 is the visually most complex stimulus in the study design. Similar to the SERP "iPod shopping" only few mouse clicks are on the ad on the top of the SERP (two clicks = 6%). Most clicks are on the Google Flight Search "sponsored links" area (43%), followed by the "organic links" area (31%). The "alternative search sites" had little visual attention with correspondingly only four clicks (11%) and the ads on the right side just three clicks (9%). Next to the amount of mouse clicks, the total visual attention (2.03 seconds) as well as the very small time to first fixation   23  
  24. 24. (0.10 seconds) indicates that the Google Flight Search area is the most attractive one of this SERP. It is remarkable that even the organic links at the bottom of the Page get more visual attention (1.16 seconds) and faster (3.35 seconds) than the "alternative search sites" (duration 0.91 seconds; time to first fixation = 5.54 seconds). Finally all target pictures suggest that the implemented "Sponsored (i)" labels do not reach the beholder’s visual attention. This finding is supported by the fact that those labels were not clicked.   24  
  25. 25. 5. References Byrne, M.D., Anderson, J.R., Douglass, S. and Matessa, M. (1999). Eye tracking the visual search of click-down menus. In Human Factors in Computing Systems: CHIʼ99 Conference Proceedings, ACM Press, New York, pp. 402-409. Duchowski, A.(2007). Methodology. Theory Springer, London. Eye and Tracking Practice. Felix, R., Hink, W. and Minor, M. (2001). Hereʼs looking at you kid: Eye-tracking applications in consumer information processing research. In Proceedings of the Annual Meeting of the Society for Marketing Advances, pp. 157-161. Ferreira, P., Rita, P., Morais, P., Oliveira, J., Gamito, P., Santos, N., Soares, F. and Sottomayor, C. (2011). Grabbing attention while reading website pages: the influence of verbal emotional cues in advertising. Journal of Eyetracking, Visual Cognition and Emotion, 1, 1, pp. 64-68. Graham, D.J. and Jefferey, R.W. (2011). Location, Location, Location: Eye-Tracking Evidence that Consumers Preferentially View Prominently Positioned Nutrition Information. Journal of the American Diet Association. 111, 11, 1704-1711. doi:   25  
  26. 26. 10.1016/j.jada.2011.08.005 Just, M.A. and Carpenter, P.A. (1980). A theory of reading: from eye fixation to comprehension. Psychological Review, 87, 4, pp. 329–354. Möller, C. and Schierl, T. (2012). Attention and selection behavior on "universal search" result pages. Journal of Eyetracking, Visual Cognition and Emotion, 2, 1, pp. 1-10.   26  
  27. 27. Institute of Communication and Research German Sport University Cologne Am Sportpark Müngersdorf 6 50933 Cologne Germany Media Contact: Dr. Carsten Möller +49(0)221 4982 6243   27