Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Towards Neuro–Information Science


Published on

Gwizdka, J. Cole, M. (2012). Towards Neuro–Information Science. Proceedings of Gmunden Retreat on NeuroIS 2012. June 3-6, 2012. Gmunden, Austria

Published in: Education
  • Be the first to comment

Towards Neuro–Information Science

  1. 1. Towards Neuro–Information Science Jacek Gwizdka & Michael Cole iSchool @ Rutgers University, NJ, USA June 5, 2012
  2. 2. Information Science  Another IS   Information Science is about : ◦  understanding information seeking behavior (why/how/where/…) ◦  helping people find information they need 2  
  3. 3. Information Systems vs. iScience  A lot of common concerns and constructs: ◦  information is digital  accessed via information systems ◦  technology – task – individual ◦  IT usefulness, user interface design, usability … ◦  trust … ◦  decision making … ◦  affective and cognitive factors ◦  information search (e.g., stopping behavior) … 3  
  4. 4. Information Systems vs. iScience  Also new opportunities: ◦  neural-correlates of constructs specific to Information Science ◦  Information Relevance : most commonly refers to topical relevance or aboutness, that is: to what extent the content of a search result matches the topic of the query or a person’s information need (e.g., Saracevic, 2007)   relevance judgment  decision making    information stopping 4  
  5. 5. Opportunities for Neuroscience to Inform ISSeven opportunities for cognitive neuroscience to inform IS research:1.  localize the neural correlates of IS constructs to better understand their nature and dimensionality;2.  complement existing sources of IS data with neuroscientific data;3.  capture hidden (automatic) processes that are difficult to measure with existing measurement methods;4.  identify antecedents of IS constructs by exploring the specifics of how IT stimuli (e.g., the design of graphical user interfaces) are processed by the brain;5.  test the outcomes of IS constructs by showing how brain activation predicts behavior (e.g., decisions);6.  infer causality among IS constructs by examining the timing of brain activations due to a common stimulus;7.  challenge existing IS assumptions and enhance IS theories that do not correspond to the brain’s functionality(Dimoka et al. 2010) 5  
  6. 6. Neuroscience and Information Science?  Eye-tracking +++  Galvanic skin response (GSR) ++  Heart –rate variability (HRV) +  EEG  fNIRS  fMRI
  7. 7. Recent and Current Projects1.  eye-tracking: modeling reading + cognitive effort2.  fMRI + eye-tracking: information relevance 7  
  8. 8. Part I: Eye-tracking  General research goal: infer and predict mental states and context of a person engaged in interactive information searching  Influence system design  adaptive systems Macro   user  task  characteris�cs,  cogni�ve  effort,  domain  knowledge   Meso   reading  pa�erns   Micro   eye-­‐gaze  posi�ons  +  �ming   8  
  9. 9. Eye-movement Presentation 9  
  10. 10. Eye-tracking Data à Patterns State1   State2   State3   10  
  11. 11. Eye-movement Patterns  New methodology to analyze eye-movement patterns ◦  Model reading and Measure cognitive effort ◦  Correlate with higher-level constructs user task characteristics, user knowledge, etc. 11  
  12. 12. Reading Model Origins  Based on E-Z Reader model Rayner , Pollatsek, Reichle ◦  Serial reading ◦  Words can be identified in parafovial region ◦  Early lexical access (word familiarity) + Complete lexical processing (word identification) 2o (70px) foveal region parafoveal regionMORE… 12  
  13. 13. Two-State Reading Model q isolated fixation fixations sequences p Read   Scan   1-p 1-q ◦  Filter fixations < 150ms (min time required for lexical processing) ◦  Model states characterized by:   probability of transitions; number of lexical fixations; duration   length of eye-movement trajectory, amount of text coveredMORE… 13  
  14. 14. Example Reading SequenceReading sequence:Fixation model states: (F F F) S (F R F) S S S S (F F FR F F) F R F F FFFF F S Reading state – R | Scanning state – S 14  
  15. 15. Cognitive Effort Measures of Reading   Reading Speed foveal region regression   Fixation Regression a b c d   Perceptual Span Perceptual span = Mean(a,b,c,d)   Fixation Duration excess (“lexical processing excess”) 15  
  16. 16. User Study 1: Cognitive Effort and Tasks Journalists’ Information Search OBI:  advanced  obituary   INT:  interview  prepara�on   CPE:  copy  edi�ng   BIC:  background  informa�on   N = 32  Do the cognitive effort measures correlate with: task difficulty (by design), observable search effort, user’s subjective perception of task difficulty  Can we detect differences between task characteristics from eye-movement patterns?MORE… 16  
  17. 17. Eye-data and Cognitive Effort Measures Subjective Task Cognitive effort measures DifficultyTask difficulty derived from eye-trackingby design reading speed Copy Editing (CPE) mean fixation duration Advance Obituary (OBI) perceptual span total fixation regressions CPE        INT      BIC        OBI   As expected: Copy Editing CPE easiest Search effort Advance Obituary OBI most difficult task time Sig: Kruskal-Wallis χ2 =46.1, p<.0001 pages visited queries entered 17  
  18. 18. Eye-data and Task Characteristics q p Read   Scan   1-p 1-q Interview  prepara�on Copy  Edi�ng   Measure   Related  Task  Characteris�cs   Frequency   Advanced  obituary  and  Interview  prepara�on  tasks:     SR bias  to  read   search  for  document;  task  goal  not  specific   of  reading   state   Copy  Edi�ng  task:  search  for  segment  and  task  goal   transi�ons   RS bias  to  scan   specific  MORE… 18  
  19. 19. Summary: Eye-tracking Methodology  Domain independent ◦  Document content is not involved  Culturally*and individually independent  Real-time modeling of user and tasks is possible  Adaptive systems feasible  Eye-tracking is coming to us! Tobii 19  
  20. 20. Part II: Current fMRI+eye-tracking Study  Information Relevance : refers to topical relevance or aboutness, that is: to what extent the content of a document (webpage) matches the topic of the query or a person’s information need (e.g., Saracevic, 2007) ◦  Relevance multi-dimensional: topical, meaningful, useful, trust, affective…  Neuralcorrelates of topical relevance judgments  Hypothesis ◦  Brain regions that are activated when relevant information is found are different from regions activated when no relevant info is found and when person does “low-level” visual word search (orthographic matching)   but no hypothesis in a sense where the brain activity is located  Exploratory research  (also: a similar experiment with eye-tracking, EEG, GSR) 20  
  21. 21. fMRI + eye-tracking lab  Lab Equipment: ◦  fMRI: 3T Siemens TRIO ◦  eye-tracker: Eyelink-1000   non-ferromagnetic optimized design; up to 2000 Hz sampling rate 21  
  22. 22. fMRI + eye-tracking 22  
  23. 23. fMRI + eye-tracking projected screen mirror eye-trackerEye-tracking imposes additional constraints on projection (geometry) 23  
  24. 24. Current Experimental Design  Two blocks (types of tasks, balanced) ◦  WS – word search: find target word in a short news story – press yes/no ◦  IS – information search: find information that answers given question – press yes/no. Three types of trials: relevant (R), topical (T), irrelevant (I) ◦  TR cycle: 2s 21 x 30s 4s 6s 4s 20s max xmx ssms nsns snsns 4s WS task target: jsdjsd djdjd djdj dkke ekek + + + kdkddk dkdkdk dkdkdkd instruc- kkdkd d d dd d djdj djdjdj word rjrjr rjr jweje ejejej ejej tions kekekek ekeke wej e eej eje j 21 x 30s 4s 8s + 20s max + 4s + 20s max xmx ssms nsns snsns + 4s + 20s max + IS task xmx ssms nsns snsns target: target: target: jsdjsd ke ekek dkdkdkkd jsdjsd djdjd djdj dkke ekek xmx ssms nsns snsns + dkdkdkkd jsdjsd ke ekek dkdkdkk kdkddk dkdkdk dkdkdkd kdkddk dkdkdkdkdkdkd instruc- kkdkd d rjr jweje ejejej kdkddk dkdkdk dkdkdkd kkdkd d rjr jweje ejeje info info info ejej kkdkd d d dd d djdj djdjdj tions rjrjr rjjweje ejejej ejej ekeke wej e ejej fjfjf fjfjfjfjf kekekek ekeke wee ejej fjfjrjr rreje j fjfjf fjfjfjfjf fjfjrjr rreje j kek ekeke wej e ejej eje j 24  
  25. 25. Planned Analysis  Two blocks (types of tasks, balanced) ◦  WS – word search: find target word in a short news story ◦  IS – information search: find information that answers given question – Three types of trials: relevant (R), topical (T), irrelevant (I)  The main contrasts of interests are: ◦  IS-R - WS ◦  IS-R - IS-T ◦  IS-R - IS-I 25  
  26. 26. A Very, Very Preliminary Analysis  For one participant, aggregated for all trials in each of two blocks (tasks)  Word search (WS)  Information Search – Relevant (IS-R) 26  
  27. 27. Stay Tuned for Results…
  28. 28. Neuro – Information ScienceAcknowledgements: Funding: Google, HP, IMLS (now funded by IMLS CAREER) Collaborators: Drs. Nicholas Belkin, Art Chaovalitwongse (U Wash), Xiangmin Zhang, Ralf Bierig (Post Doc); PhD students: Michael Cole (co-author), Chang Liu, Jingjing Liu, Irene Lopatovska + many Master and undergraduate students … 28  
  29. 29. Fragen?More info & contact 29