Learning about Information Searchers from Eye-Tracking by Jacek Gwizdka


Published on

Presented by Jacek Gwizdka at University of Missouri

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Tasks varied in several dimensions: complexity defined as the number of necessary steps needed to achieve the task goal (e.g. identifying an expert and then finding their contact information), the task product (factual vs. intellectual, e.g. fact checking vs. production of a document), the information object (a complete document vs. a document segment), andthe nature of the task goal (specific vs. amorphous).
  • Tasks varied in several dimensions: complexity defined as the number of necessary steps needed to achieve the task goal (e.g. identifying an expert and then finding their contact information), the task product (factual vs. intellectual, e.g. fact checking vs. production of a document), the information object (a complete document vs. a document segment), andthe nature of the task goal (specific vs. amorphous).
  • Eye tracking work on reading behavior in information search have mostly analyzed eye gaze position aggregates ('hot spots').This does not address the fixation sub-sequences that are true reading behavior.
  • Reading models can be built on the fly They only requires analysis of the recent eye movement sequence to classify the observed fixations
  • Learning about Information Searchers from Eye-Tracking by Jacek Gwizdka

    1. 1. Jacek Gwizdka<br />Department of Library and Information Science<br />School of Communication and Information<br />Rutgers University<br />Monday, April 4, 2011<br />Learning about Information Searchers from Eye-Tracking<br />CONTACT: <br />www.jsg.tel<br />
    2. 2. Outline<br />Overall research goals<br />Eye-tracking – fundamentals<br />Eye-fixation patterns: reading models (Exp 1; Exp 3)<br />Search results presentation and cognitive abilities (Exp 2)<br />Summary and Challenges<br />2<br />
    3. 3. Overall Research Goals<br />Characterization and enhancement of human information interaction mediated by computing technology <br />Characterization: cognitive and affective user states –traditionally: little access to the mental/emotional states of users while they are engaged in the search process<br />Implicit data collection about searchers’ cognitive and affective states in relation to information search phases<br />Enhancement: Personalization and Adaptation<br />3<br />
    4. 4. Example: Implicit Characterization of Cognitive Load on Web Search<br />4<br />higher average<br />cognitive load: Q & B<br />35%<br />27%<br />higher peak <br />cognitive load: C<br />START<br />Q<br />formulate <br />query<br />Lview search results list<br />B <br />bookmark<br />page<br />Cview <br />content<br />page<br />END<br />97%<br />58%<br />30%<br />42%<br />7%<br />95%<br />(Gwizdka, JASIST, 2010)<br />
    5. 5. Eye-Tracking?<br />Early attempts late XIX c.; early 1950’s - using a movie camera and hand-coding (Fitts, Jones & Milton 1950)<br />Now computerized and “easy to use” <br />infrared light sources and cameras<br />stationary and mobile<br />5<br />Current Tobii eye-trackers<br />
    6. 6. Eye-tracking – fundamental assumptions<br />Top-down vs. bottom-up control <br />in between: language processing (higher-level) controls when eyes move, visual processing (lower-level) controls where eyes move(Reichle et al., 1998)<br />Eye-mind link hypothesis: attention is where eyes are focused (Just & Carpenter, 1980; 1987)<br />Overt and covert attention<br />Attention can move with no eye movement BUT eyes cannot move without attention<br />6<br />
    7. 7. Data from Eye-tracking Devices<br />eye gaze points <br />eye gaze points in screen coordinates + distance <br />eye fixations in screen coordinates + validity <br />pupil diameter <br />[head position 3D, distance from monitor]<br />50/60Hz; 300Hz; 1000-2000Hz eye-trackers<br />common: 60Hz: one data record every 16.67ms<br />7<br />Tobii T-60 <br />eye-tracker<br />
    8. 8. Eye-Tracking Can … <br />Eye tracking can allow identification of the specific content acquired by the person from Web pages <br />Eye tracking enables high resolution analysis of searcher’s activity during interactions with information systems<br />And more…<br />8<br />Example: composing answer and <br />from information on a Web page<br />(video) <br />
    9. 9. Related Work in Information Science<br />Interaction with search results<br />Interaction with SERPs (Granka et al., 2004; Lorigo et al., 2007; 2008)<br />Effects results presentation (Cutrell et al., 2007; Kammerer al., 2010)<br />Relevance detection (Buscher, et al. 2009)<br />Implicit Feedback (Fu, X., 2009); Query expansion (Buscher, et al. 2009)<br />Relevance detection <br />Pupillometry (Oliveira, Aula, Russell, 2009)<br />Detection of task differences from eye-gaze patterns<br />Reading/reasoning/search/object manipulation (Iqbal & Bailey, 2004)<br />Informational vs. transactional tasks (Terai , et al., 2008)<br />Task detection is also one of our research interests<br />9<br />
    10. 10. Experiment 1: Journalism tasks – Open Web Search<br />32 journalism students<br />4 journalistic tasks (realistic, created by journalism faculty and journalists)<br />Tasks: <br />advanced obituary (OBI) <br />interview preparation (INT)<br />copy editing (CPE)<br />background information (BIC)<br />10<br />Task facets:<br /><ul><li>product: factual vs. intellectual
    11. 11. level: whole document vs. segment
    12. 12. nature of task goal
    13. 13. complexity – number of steps needed</li></ul>Note: OBI vs. CPE are most dissimilar<br />
    14. 14. Experiment 1 – Research Questions<br />Can we detect task type (differences in task facets) from implicit interaction data (e.g., eye-tracking) ?<br />How do we aggregate information from eye-tracking data?<br />11<br />
    15. 15. Eye-gaze patterns<br />Eye-tracking research have frequently analyzed eye-gaze position aggregates ('hot spots’)<br />spatiotemporal-intensity – heat maps<br />also sequential – scan paths<br />12<br /><ul><li> Higher-order patterns:
    16. 16. reading models </li></li></ul><li>Reading Eye Patterns<br />Reading and scanning have easily distinguished patterns of fixations and saccades. (Rayner & Fischer, 1996)<br />Lexical Processing of Words<br />Reading research has established word availability is a function of fixation duration:<br />Orthographic recognition: 40-50 ms<br /> time to move data from eyes to mind<br />Phonological recognition: 55-70ms<br />Lexical availability (typical): 113 ms – 150ms (Rayner, 1998)<br />Unfamiliar or complex meanings require longer processing<br />Eyes do not saccade until the word has been processed<br />13<br />
    17. 17. Scan Fixations vs. Reading Fixations<br />Scanning fixations provide some semantic information, limited to foveal(1° visual acuity) visual field (Rayner & Fischer, 1996)<br />Fixations in a reading sequence provide more information than isolated “scanning” fixations:<br />information is gained from the larger parafoveal (5° beyond foveal focus) region (Rayner et al., 2003) (asymmetrical, in dir of reading)<br />richer semantic structure available from text compositions (sentences, paragraphs, etc.) <br />Some of the types of semantic information available only through reading sequences may be crucial to satisfy task requirements.<br />14<br />
    18. 18. Reading Models<br />We implemented the E-Z Reader reading model (Reichle et al., 2006)<br />Inputs: (eye fixation location, duration)<br />Fixation duration >113 ms – threshold for lexical processing (Reingold & Rayner, 2006)<br />The algorithm distinguishes reading fixation sequences from isolated fixations, called 'scanning' fixations<br />Each lexical fixation is classified to (S,R) (Scan, Reading)<br />These sequences used to create a state model<br />15<br />
    19. 19. Reading Model – States and Characteristics<br />Two states: transition probabilities<br />Number of lexical fixations and duration<br />16<br />
    20. 20. Example Reading Sequence<br />17<br />
    21. 21. Results: Search Task Effect on Reading/Scanning<br />Task effects on transition probabilities SR & RS(all subjects & pages)<br />18<br /><ul><li>For OBI, INT searchers biased to continue reading
    22. 22. For CPE to continue scanning</li></ul>Searchers are adopting different reading strategies for different task types<br />(Cole, Gwizdka, Liu, Bierig, Belkin & Zhang, 2010)<br />
    23. 23. Results: Search Task Facets and Text Acquisition<br />For highly attended pages<br />19<br />Total Text Acquisition on SERPs and Content <br />per page<br />Total Text Acquired on SERPs and Content<br />
    24. 24. Results: Search Task Facets and State Transitions <br />For highly attended pages<br />20<br />Read  Scan<br />Scan Read<br />Scan Read<br />Read  Scan<br />State Transitions on Content pages per page<br />State Transitions on SERPs per page<br />
    25. 25. Task Facets Effects - Summary<br />For highly attended pages<br />21<br />(Cole, Gwizdka, Liu, Bierig, Belkin & Zhang, submitted, 2011)<br />
    26. 26. Scan<->Read Transition Probabilities in 2 Experiments<br />Person’s tendency to readscan related to scanread? (i.e., is p related to q ?)<br />p ~ 1-q<br />Genomics tasks (N=40)<br />Journalistic tasks (N=32)<br />correlation (Spearman ρ): 0.914 and 0.830 <br />
    27. 27. Experiment 1: Conclusions<br />Searchers’ reading / scanning behavior affected by task<br />Tasks facets can be “detected” from eye-tracking data (from reading model properties)<br />Reading models can be built on the fly (during search)  real-time observations of eye movements can be used by adaptive search systems<br />Challenge: Lack of baseline data about reading models of individuals <br />23<br />
    28. 28. Experiment 2: Result List vs. Overview Tag-Cloud<br />37 participants<br />Everyday information seeking tasks (travel, shopping…)<br /> - two levels of task complexity<br />Two user interfaces<br />24<br />2. Overview UI <br />(Tag Cloud)<br />1. List UI<br />
    29. 29. Experiment 2: User Actions in Two Interfaces<br />25<br />1. List<br />2. Overview <br />Tag Cloud<br />
    30. 30. Experiment 2: Research Questions<br />Does the search results overview benefit users?<br />Task effects?<br />Individual differences - cognitive ability effects?<br />26<br />
    31. 31. General Results<br />Search results overview (“tag cloud”) benefited users <br />made them faster<br />facilitated formulation of more effective queries<br />More complex tasks were indeed more demanding – required more search effort <br />27<br />(Gwizdka, Information Research, 2009)<br />
    32. 32. Task and UI and Reading Model differences<br />Complex tasks required more reading effort<br />Longer max reading fixation length and more reading fixation regressions <br />Overview UI required less effort<br />Scanning more likely (S-S higher; S-R lower; R-S higher)<br />Total reading scan path length shorter but total scan path (including scanning) were longer<br />Less and shorter mean fixations per page visited<br />28<br />List Overview<br />
    33. 33. Task and UI Interaction and Reading model data<br />For complex tasks UI effect <br />Higher probability of short reading sequences in Overview UI<br />For simple tasks UI effect<br />Shorter length of reading scan paths per page and less fixations per page<br />Task & UI interaction <br />Speed of reading: <br />for complex tasks faster reading in Overview than in List UI<br />for simple tasks faster in List than in Overview UI<br />29<br />
    34. 34. User Interface Features – Individual Differences<br />Two users, same UI and task<br />30<br />
    35. 35. Individual Differences – Least Effort? <br />Higher cognitive ability searchers were faster in Overview UI and on simple tasks (same number of queries)<br />Higher ability searchers did more in more demanding situations<br />higher search effort did not seem to improve task outcomes <br />31<br />For task complexity factor and working memory (WM)<br />F(144,1)=4.2; p=.042<br />F(144,1)=3.1; p=.08<br />
    36. 36. Task and Working Memory – Eye-tracking Data <br />High WM less likely to keep scanning<br />High WM higher reading speed (scan path/total fixation duration)<br />Number and duration of reading sequences differs <br />(borderline: 0.05<p<0.1)<br />For high WM searchers: <br />for complex more reading<br />for simple tasks less reading <br />For low WM no such difference!<br />32<br />
    37. 37. Experiment 2: Conclusions<br />Overview UI was faster – reflected in some eye-tracking measures<br />Task complexity differences reflected in some eye-tracking measures<br />Some effects of cognitive abilities on interaction<br />e.g., task & high WM – more effort than needed <br />opportunistic discovery of information?<br />“violation” of the least effort principle not fully explained yet<br />33<br />
    38. 38. 34<br />
    39. 39. Current Project: Can We Implicitly Detect Relevance Decisions?<br /><ul><li>Implicit characterization of Information Search Process using physiological devices
    40. 40. Can we detect when searchers make information relevance decisions?</li></ul>Emotiv EPOC <br />wireless EEG headset<br />EEG<br />Start with pupillometry<br />info relevance (Oliveria, Russell, Aula, 2009)<br />low-level decision timing (Einhäuser, et al. 2010)<br />Also look at EEG, GSR<br />Funded by Google Research Award <br />pupil animation<br />Eye tracking<br />Tobii T-60 <br />eye-tracker<br />GSR<br />
    41. 41. Summary & Conclusions<br />Eye tracking enables high resolution analysis of searcher’s activity during interactions with information systems<br />There is more beyond eye-gaze locations with timestamps <br />Eye-tracking data:<br />can support for identification of search task types<br />reflects differences in searcher performance on user interfaces<br />reflects individual differences between searchers<br />High potential for implicit detection of a searcher’s states<br />36<br />
    42. 42. Some Challenges<br />High-resolution data (low-level)<br />How do we create higher-level patterns?<br />How do we detect them computationally?<br />How do we deal with ind. diffs(baseline data)?<br />37<br />(Iqbal & Bailey, 2004)<br />(Terai et al., 2008)<br />(Lorigo et al., 2008)<br />
    43. 43. High-resolution Eye-tracking is Coming Soon to You<br />Eye tracking technology is declining in price and in 2-3 years could be part of standard displays.<br />Already in luxury cars and semi-trucks (sleep detection)<br />Computers with built in eye-tracking <br />38<br />Tobii / Lenovo<br />proof of concept eye-tracking <br />laptop - March 2011<br />
    44. 44. Thank you! Questions?<br />Jacek Gwizdka contact: http://jsg.tel<br />PoODLEProject: Personalization of the Digital Library Experience<br /> IMLS grant LG-06-07-0105-07<br />http://comminfo.rutgers.edu/research/poodle<br />or for short: http://bit.ly/poodle_project<br />PoODLE PIs: Nicholas J. Belkin, Jacek Gwizdka, Xiangmin Zhang<br /> Post-Doc: Ralf Bierig, <br /> PhD Students: Michael Cole (Reading Models + E-Z Reader algorithm), <br /> Jingjing Liu, (now Asst Prof.), Chang Liu<br />