Towards  Neuro–Information Science      Jacek Gwizdka & Michael Cole          iSchool @ Rutgers University, NJ, USA       ...
Information Science  Another  IS   Information Science is about : ◦  understanding information seeking behavior (why/ho...
Information Systems vs. iScience  A lot of common concerns and constructs:   ◦  information is digital  accessed via inf...
Information Systems vs. iScience  Also new opportunities:   ◦  neural-correlates of constructs specific to Information Sc...
Opportunities for Neuroscience to Inform ISSeven opportunities for cognitive neuroscience to inform IS research:1.  locali...
Neuroscience and Information Science?  Eye-tracking  +++  Galvanic skin response (GSR) ++  Heart –rate variability (HRV...
Recent and Current Projects1.    eye-tracking: modeling reading + cognitive effort2.    fMRI + eye-tracking: information r...
Part I: Eye-tracking  General  research goal: infer and predict mental states and   context of a person engaged in intera...
Eye-movement Presentation                            9	  
Eye-tracking Data à Patterns             State1	                  State2	                            State3	             ...
Eye-movement Patterns  New   methodology to analyze eye-movement patterns ◦  Model reading and Measure cognitive effort ◦...
Reading Model Origins  Based     on E-Z Reader model Rayner , Pollatsek, Reichle ◦  Serial reading ◦  Words can be identi...
Two-State Reading Model                                             q                                                     ...
Example Reading SequenceReading sequence:Fixation model states:   (F F F) S (F R F) S S S S (F F FR F F) F                ...
Cognitive Effort Measures of Reading    Reading     Speed                                   foveal region                ...
User Study 1: Cognitive Effort and Tasks        Journalists’        Information Search        OBI:	  advanced	  obituary	 ...
Eye-data and Cognitive Effort Measures                                                              Subjective Task       ...
Eye-data and Task Characteristics                                                               q                         ...
Summary: Eye-tracking Methodology  Domain   independent ◦  Document content is not involved  Culturally*and individually...
Part II: Current fMRI+eye-tracking Study  Information Relevance : refers to topical relevance or aboutness, that is: to w...
fMRI + eye-tracking lab  Lab Equipment:   ◦  fMRI: 3T Siemens TRIO   ◦  eye-tracker: Eyelink-1000     non-ferromagnetic ...
fMRI + eye-tracking                      22	  
fMRI + eye-tracking                                                                projected                              ...
Current Experimental Design  Two blocks (types of tasks, balanced)   ◦  WS – word search: find target word in a short new...
Planned Analysis  Two blocks (types of tasks, balanced)   ◦  WS – word search: find target word in a short news story   ◦...
A Very, Very Preliminary Analysis  For     one participant, aggregated for all trials in each of two blocks (tasks)  Wor...
Stay Tuned for Results…
Neuro – Information ScienceAcknowledgements: Funding: Google, HP, IMLS (now funded by IMLS CAREER)                  Collab...
Fragen?More info & contact http://jsg.tel                                     29	  
Upcoming SlideShare
Loading in …5
×

Towards Neuro–Information Science

497 views

Published on

Gwizdka, J. Cole, M. (2012). Towards Neuro–Information Science. Proceedings of Gmunden Retreat on NeuroIS 2012. June 3-6, 2012. Gmunden, Austria

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
497
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Towards Neuro–Information Science

  1. 1. Towards Neuro–Information Science Jacek Gwizdka & Michael Cole iSchool @ Rutgers University, NJ, USA jacek@neuroinfoscience.org http://jsg.tel June 5, 2012
  2. 2. Information Science  Another IS   Information Science is about : ◦  understanding information seeking behavior (why/how/where/…) ◦  helping people find information they need 2  
  3. 3. Information Systems vs. iScience  A lot of common concerns and constructs: ◦  information is digital  accessed via information systems ◦  technology – task – individual ◦  IT usefulness, user interface design, usability … ◦  trust … ◦  decision making … ◦  affective and cognitive factors ◦  information search (e.g., stopping behavior) … 3  
  4. 4. Information Systems vs. iScience  Also new opportunities: ◦  neural-correlates of constructs specific to Information Science ◦  Information Relevance : most commonly refers to topical relevance or aboutness, that is: to what extent the content of a search result matches the topic of the query or a person’s information need (e.g., Saracevic, 2007)   relevance judgment  decision making    information stopping 4  
  5. 5. Opportunities for Neuroscience to Inform ISSeven opportunities for cognitive neuroscience to inform IS research:1.  localize the neural correlates of IS constructs to better understand their nature and dimensionality;2.  complement existing sources of IS data with neuroscientific data;3.  capture hidden (automatic) processes that are difficult to measure with existing measurement methods;4.  identify antecedents of IS constructs by exploring the specifics of how IT stimuli (e.g., the design of graphical user interfaces) are processed by the brain;5.  test the outcomes of IS constructs by showing how brain activation predicts behavior (e.g., decisions);6.  infer causality among IS constructs by examining the timing of brain activations due to a common stimulus;7.  challenge existing IS assumptions and enhance IS theories that do not correspond to the brain’s functionality(Dimoka et al. 2010) 5  
  6. 6. Neuroscience and Information Science?  Eye-tracking +++  Galvanic skin response (GSR) ++  Heart –rate variability (HRV) +  EEG  fNIRS  fMRI
  7. 7. Recent and Current Projects1.  eye-tracking: modeling reading + cognitive effort2.  fMRI + eye-tracking: information relevance 7  
  8. 8. Part I: Eye-tracking  General research goal: infer and predict mental states and context of a person engaged in interactive information searching  Influence system design  adaptive systems Macro   user  task  characteris�cs,  cogni�ve  effort,  domain  knowledge   Meso   reading  pa�erns   Micro   eye-­‐gaze  posi�ons  +  �ming   8  
  9. 9. Eye-movement Presentation 9  
  10. 10. Eye-tracking Data à Patterns State1   State2   State3   10  
  11. 11. Eye-movement Patterns  New methodology to analyze eye-movement patterns ◦  Model reading and Measure cognitive effort ◦  Correlate with higher-level constructs user task characteristics, user knowledge, etc. 11  
  12. 12. Reading Model Origins  Based on E-Z Reader model Rayner , Pollatsek, Reichle ◦  Serial reading ◦  Words can be identified in parafovial region ◦  Early lexical access (word familiarity) + Complete lexical processing (word identification) 2o (70px) foveal region parafoveal regionMORE… 12  
  13. 13. Two-State Reading Model q isolated fixation fixations sequences p Read   Scan   1-p 1-q ◦  Filter fixations < 150ms (min time required for lexical processing) ◦  Model states characterized by:   probability of transitions; number of lexical fixations; duration   length of eye-movement trajectory, amount of text coveredMORE… 13  
  14. 14. Example Reading SequenceReading sequence:Fixation model states: (F F F) S (F R F) S S S S (F F FR F F) F R F F FFFF F S Reading state – R | Scanning state – S 14  
  15. 15. Cognitive Effort Measures of Reading   Reading Speed foveal region regression   Fixation Regression a b c d   Perceptual Span Perceptual span = Mean(a,b,c,d)   Fixation Duration excess (“lexical processing excess”) 15  
  16. 16. User Study 1: Cognitive Effort and Tasks Journalists’ Information Search OBI:  advanced  obituary   INT:  interview  prepara�on   CPE:  copy  edi�ng   BIC:  background  informa�on   N = 32  Do the cognitive effort measures correlate with: task difficulty (by design), observable search effort, user’s subjective perception of task difficulty  Can we detect differences between task characteristics from eye-movement patterns?MORE… 16  
  17. 17. Eye-data and Cognitive Effort Measures Subjective Task Cognitive effort measures DifficultyTask difficulty derived from eye-trackingby design reading speed Copy Editing (CPE) mean fixation duration Advance Obituary (OBI) perceptual span total fixation regressions CPE        INT      BIC        OBI   As expected: Copy Editing CPE easiest Search effort Advance Obituary OBI most difficult task time Sig: Kruskal-Wallis χ2 =46.1, p<.0001 pages visited queries entered 17  
  18. 18. Eye-data and Task Characteristics q p Read   Scan   1-p 1-q Interview  prepara�on Copy  Edi�ng   Measure   Related  Task  Characteris�cs   Frequency   Advanced  obituary  and  Interview  prepara�on  tasks:     SR bias  to  read   search  for  document;  task  goal  not  specific   of  reading   state   Copy  Edi�ng  task:  search  for  segment  and  task  goal   transi�ons   RS bias  to  scan   specific  MORE… 18  
  19. 19. Summary: Eye-tracking Methodology  Domain independent ◦  Document content is not involved  Culturally*and individually independent  Real-time modeling of user and tasks is possible  Adaptive systems feasible  Eye-tracking is coming to us! Tobii 19  
  20. 20. Part II: Current fMRI+eye-tracking Study  Information Relevance : refers to topical relevance or aboutness, that is: to what extent the content of a document (webpage) matches the topic of the query or a person’s information need (e.g., Saracevic, 2007) ◦  Relevance multi-dimensional: topical, meaningful, useful, trust, affective…  Neuralcorrelates of topical relevance judgments  Hypothesis ◦  Brain regions that are activated when relevant information is found are different from regions activated when no relevant info is found and when person does “low-level” visual word search (orthographic matching)   but no hypothesis in a sense where the brain activity is located  Exploratory research  (also: a similar experiment with eye-tracking, EEG, GSR) 20  
  21. 21. fMRI + eye-tracking lab  Lab Equipment: ◦  fMRI: 3T Siemens TRIO ◦  eye-tracker: Eyelink-1000   non-ferromagnetic optimized design; up to 2000 Hz sampling rate 21  
  22. 22. fMRI + eye-tracking 22  
  23. 23. fMRI + eye-tracking projected screen mirror eye-trackerEye-tracking imposes additional constraints on projection (geometry) 23  
  24. 24. Current Experimental Design  Two blocks (types of tasks, balanced) ◦  WS – word search: find target word in a short news story – press yes/no ◦  IS – information search: find information that answers given question – press yes/no. Three types of trials: relevant (R), topical (T), irrelevant (I) ◦  TR cycle: 2s 21 x 30s 4s 6s 4s 20s max xmx ssms nsns snsns 4s WS task target: jsdjsd djdjd djdj dkke ekek + + + kdkddk dkdkdk dkdkdkd instruc- kkdkd d d dd d djdj djdjdj word rjrjr rjr jweje ejejej ejej tions kekekek ekeke wej e eej eje j 21 x 30s 4s 8s + 20s max + 4s + 20s max xmx ssms nsns snsns + 4s + 20s max + IS task xmx ssms nsns snsns target: target: target: jsdjsd ke ekek dkdkdkkd jsdjsd djdjd djdj dkke ekek xmx ssms nsns snsns + dkdkdkkd jsdjsd ke ekek dkdkdkk kdkddk dkdkdk dkdkdkd kdkddk dkdkdkdkdkdkd instruc- kkdkd d rjr jweje ejejej kdkddk dkdkdk dkdkdkd kkdkd d rjr jweje ejeje info info info ejej kkdkd d d dd d djdj djdjdj tions rjrjr rjjweje ejejej ejej ekeke wej e ejej fjfjf fjfjfjfjf kekekek ekeke wee ejej fjfjrjr rreje j fjfjf fjfjfjfjf fjfjrjr rreje j kek ekeke wej e ejej eje j 24  
  25. 25. Planned Analysis  Two blocks (types of tasks, balanced) ◦  WS – word search: find target word in a short news story ◦  IS – information search: find information that answers given question – Three types of trials: relevant (R), topical (T), irrelevant (I)  The main contrasts of interests are: ◦  IS-R - WS ◦  IS-R - IS-T ◦  IS-R - IS-I 25  
  26. 26. A Very, Very Preliminary Analysis  For one participant, aggregated for all trials in each of two blocks (tasks)  Word search (WS)  Information Search – Relevant (IS-R) 26  
  27. 27. Stay Tuned for Results…
  28. 28. Neuro – Information ScienceAcknowledgements: Funding: Google, HP, IMLS (now funded by IMLS CAREER) Collaborators: Drs. Nicholas Belkin, Art Chaovalitwongse (U Wash), Xiangmin Zhang, Ralf Bierig (Post Doc); PhD students: Michael Cole (co-author), Chang Liu, Jingjing Liu, Irene Lopatovska + many Master and undergraduate students … 28  
  29. 29. Fragen?More info & contact http://jsg.tel 29  

×