The lnteiaction oi. ’ I~'aviga'tion Instructions and

Visual Attention in Dynamic Automotive Environments’

Hendrik Koesli...
Overview

Navigation in Dynamic Automotive Environments

Driver-assistant devices such as I:   FIE?   R‘. ‘F. 'P'i‘?  are ...
Paradigm

Driving while following navigation instructions

- Task

INTRODUCTION

METHOD

- A dual task?  Not really!  It i...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

Paradigm

Driving while following navigation instructions

- Challenges

— Com...
Paradigm

Driving while following navigation instructions

Errol‘;  rr. uL1. Lirqi ‘: ll‘l'| )llI‘lElIl BL: 'y li: l'n;  :...
Paradigm Focus:  Visuo-Linguistic Interaction

Attending to road scenes while Iistenin to navigation instructions

Tasks 8...
Research Oueslions & Hypotheses

Which properties of navigation instructions.  traffic situations and their
interaction pa...
Experiment Details

Participants
- n =  80 experienced drivers
- 50% male.  50% female
- driving experience:  M =  9.4 yea...
Experiment Details

Independent variables

- Visual - Scene complexity (number of vehicles & turning alternatives)

- Ling...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

Experiment Set-Up

EyeLink II eye tracker

Eye-tracking
laboratory
1. Comprehension of Instructions

Response COITBCUIESS

correctness 0/0

INTRODUCTION

METHOD

Significant variation with
...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

1. Comprehension of Instructions

Safe/ Unsafe response

siu correctness °. i‘...
1. Comprehension of Instructions

Summary

Highly correct "righti"| eft“ responses.  independent of all factors
Considerab...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

2. Distribution of Attention

Number of fixations rNF

Vehicle areas

H2: 152)...
2. Distribution of Attention

Gaze duration rGD

Vehicle areas Relevant route areas

H2.‘ 152).  145.11.‘p -:  0.001 H2: 1...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

2. Distribution of Attention

Gaze trajectories

;  I as:  

Y

(#175: ‘:1 1: ...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

2. Distribution of Attention

Gaze trajectories

3-fit ‘: g=ET', "~$}" en-7.-s:...
2. Distribution of Attention

Gaze trajectories

before instruction 

: KI 7,, 

during instruction

INTRODUCTION

METHOD
...
INTRODUCTION

METHOD

RESULTS

CONCLUSIONS

2. Distribution of Attention

Gaze trajectories

before instruction

during In...
2. Distribution of Attention

General visual processing pattern

- Before instruction
- Coarse scene overview
- Several br...
2. Distribution of Attention

Special visual processing patterns

- Inefficient visual processing

- Complex scene.  messa...
3. 3< cl.  Incremental Generation of Representations

Model:  Mapping gaze trajectories onto navigation instructions

- Pr...
3. 8< cl.  Incremental Generation of Representations

Model:  Mapping gaze trajectories onto navigation instructions

- Fi...
Summary and Conclusions

Analysis of eye movements revealed how visuo-linguistic interaction
affected visual attention

Di...
Summary and Conclusions

Perception and processing of navigation messages are greatly compromised
when visual attention mo...
Thank you for your attention! 

Hendrik Koesling & Honan 6 Reilly
I he Interaction of Navigation Instructions and Visual A...
Upcoming SlideShare
Loading in …5
×

The Interaction of Navigation Instructions and Visual Attention in Dynamic Automotive Environments

1,192
-1

Published on

Hendrik Koesling and Ronan G. Reilly
National University of Ireland Maynooth, Department of Computer Science, Maynooth, Co. Kildare, IRELAND

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,192
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Interaction of Navigation Instructions and Visual Attention in Dynamic Automotive Environments

  1. 1. The lnteiaction oi. ’ I~'aviga'tion Instructions and Visual Attention in Dynamic Automotive Environments’ Hendrik Koesling 8. Ronan G. Reilly Department of Computer Science National University of Ireland Maynooth Maynooth. Co. Kildare Ireland e-mail: hendrik@cs. nuim. ie ' Part—li. mo+; «o by the German Science Foundation -jDFG grant l<O2:l50i'1 ti
  2. 2. Overview Navigation in Dynamic Automotive Environments Driver-assistant devices such as I: FIE? R‘. ‘F. 'P'i‘? are a common feature In many modern vehicles (e. g. Barfield 8. Dingus, 1998). Processing navigation instructions, however, is a challenging task that has to be accomplished I ii. -I to the actual driving task and often distracts drivers (e. g Ftecarte 8. Nunes, 2000). A flexible rii. i:‘l-1-‘wt n’ r: tt»r= "i'ini: could optimise the I‘2:’. i']"ll'l‘v‘>" mi spent on each task and improve driving safety: In = rl~, »i-anti: driving environments, navigation messages should be adapted according to the current traffic INTRODUCTION situation. To verify this hypothesis. we investigated how attending to dynamic road scenes while following navigation messages . ~ : . ~ ii 1 * * | "lif“’E‘: ‘7i|1‘: ’] I affects message comprehension and driver perfomiance. METHOD To quantify this relation. we suggest to modify a model that successfully describes the "C’{? fl‘>’. ‘l'i’l generation of representations from spoken language in static scenes by IL‘ mapping navigation instructions onto RESULTS gaze trajectories (e. g. Allopenna et al. . 1998: Eberhard et al. , 1995: Spivey et al. , 2002). This may allow us to develop optimal properties of navigation messages for specific traffic situations. CONCLUSIONS
  3. 3. Paradigm Driving while following navigation instructions - Task INTRODUCTION METHOD - A dual task? Not really! It is a multiple task: a) Actual driving (e. g. steering, changing gears. etc. ) RESULTS . , , b) Observation of traffic, environment and mirrors c) Listening to navigation messages CONCLUSIONS
  4. 4. INTRODUCTION METHOD RESULTS CONCLUSIONS Paradigm Driving while following navigation instructions - Challenges — Complex tasks — Complex environments - Parallel processing of all tasks required - Dynamically changing environment
  5. 5. Paradigm Driving while following navigation instructions Errol‘; rr. uL1. Lirqi ‘: ll‘l'| )llI‘lElIl BL: 'y li: l'n; :lJll: -r>: d vrvwr ivirvr: -n+_*' -, . . :5:-; .:~+e - Goal WTRODUCVON - Optimise cognitive workload spent on each task in order to improve driving safety - Goal approach METHOD « Dynamic adaptation of navigation messages according to current traffic situation - Methodological approach RESULTS — Use distribution of (visual) attention as indicator for cognitive workload « Eye tracking: Recording of eye movements to monitor visual attention (e. g. Just & Carpenter. 1987) CONCLUSIONS
  6. 6. Paradigm Focus: Visuo-Linguistic Interaction Attending to road scenes while Iistenin to navigation instructions Tasks 8. Effects - Observation of traffic and environment - Listening to navigation messages - Effects on message comprehension and driver performance Key factors - Scene complexity - Linguistic structure of instruction messages INTRODUCTION . . - Message timing Accounting for driving situation dynamics METHOD - Assess visual attention distribution before, during and after instruction Relating instruction processing. visual attention and driver performance - Incremental generation of representations from spoken language (e. g. Allopenna et al. . 1998: Eberhard et al. . 1995; Spivey at al. . 2002) - Map gaze trajectories onto navigation instructions RESULTS CONCLUSIONS
  7. 7. Research Oueslions & Hypotheses Which properties of navigation instructions. traffic situations and their interaction particularly affect message understanding? How does the distribution of attention change between the different phases of driving while following navigation instructions? INTRODUCHON . Can we — analogous to the incremental generation of representations from spoken language in static scenes — serially map gaze trajectories onto navigation instructions in dynamic situations? METHOD Does a correlation exist between possible disturbances of this mapping and a deterioration of driver responses? RESULTS CONCLUSIONS
  8. 8. Experiment Details Participants - n = 80 experienced drivers - 50% male. 50% female - driving experience: M = 9.4 years (SD = 2.3) Stimuli Experimental task INTRODUCTION METHOD 1. Left or right response butto ; fipeclively (during video presentation) 2. Verbal (e. g.. “sale”; after vi i ijesention) Apparatus RESULTS - SR Research Eyelink ll eyet .4r - “VDesigner" visual programm‘? I ,1‘-‘nvironment for eye-tracking experiments (e. g. Koesling, Clermont & Rifti . "’001) A» -v. — as“ CONCLUSIONS
  9. 9. Experiment Details Independent variables - Visual - Scene complexity (number of vehicles & turning alternatives) - Linguistic ~ Message complexity (message length & number of propositions) - Verb position - Temporal — Message timing (in relation to turning point) Dependent variables INTRODUCTION - Response correctness - Number of lixations NF METHOD - Gaze duration GD - NF and GD separated for RESULTS - time intervals (before. during and after message presentation) and — spatial stimulus regions (verbal reference, relevant road furniture. vehicles and irrelevant areas) CONCLUSIONS
  10. 10. INTRODUCTION METHOD RESULTS CONCLUSIONS Experiment Set-Up EyeLink II eye tracker Eye-tracking laboratory
  11. 11. 1. Comprehension of Instructions Response COITBCUIESS correctness 0/0 INTRODUCTION METHOD Significant variation with - Scene complexity RESU Us Independent of all factors - Message complexity - Message timing CONCLUSIONS
  12. 12. INTRODUCTION METHOD RESULTS CONCLUSIONS 1. Comprehension of Instructions Safe/ Unsafe response siu correctness °. i‘c siu correctness °. i'c Scene complexity § F(1; 76) : 57.33;p < 0.001 90 80 70 60L simple comp'ex scene complexity Message issue time 100 F(1: 75) ; 77.02; p < 0001 so 80 70 60% early late message issue time s-‘u correctness °. -'2 sru correctness ’-it Message complexity F(2.‘ 152) : 6I.96.'p < 0.001 simple intermediate complex message complexity Verb position No significant effect start end verb position
  13. 13. 1. Comprehension of Instructions Summary Highly correct "righti"| eft“ responses. independent of all factors Considerable errors in "sale. /unsafe" responses. varying with - message complexity - scene complexity - message issuing time - but not with verb position | N‘| 'RODUCT|0N Significantly higher salelunsafe error rates - for complex messages - for complex scenes METHOD - when messages are issued late Interaction between message complexity, message timing and verb position [F(3; 76) = 60.42; p < 0.001] RESU LTS Sale/ unsafe decision correctness drops signilicantly for complex, late messages with verb at end of instruction CONCLUSIONS
  14. 14. INTRODUCTION METHOD RESULTS CONCLUSIONS 2. Distribution of Attention Number of fixations rNF Vehicle areas H2: 152) - 150.23; p < 0.001 rNF vehicles ti‘. before time interval (relative to msg. ) during after Instruction reference areas H2.‘ 152) ; 162.90.‘p < 0.001 rNF instruc ‘Til: before time interval (relative to rnsg. ) during after rNF route *7-‘Ia rNF irrelevant 9-2. G) C) J8 C) l) O (3 Relevant route areas H2: 152) :1 12.72;p -: 0001 before time interval (relative to msg. ) during after Irrelevant areas No significant effect before time interval (relative to msg. ) during after
  15. 15. 2. Distribution of Attention Gaze duration rGD Vehicle areas Relevant route areas H2.‘ 152). 145.11.‘p -: 0.001 H2: 152) :128.82,'p -: 0001 rGD vehicles ‘T4: rGD route ".2 before during after before during after time interval (relative to m5g. ) time interval (relative to mag‘) INTRODUCTION Instruction reference areas Irrelevant areas H2: 152) ; 165.44: p < 0.001 No significant effect METHOD rGD tnstruc = .. rGD irrelevant 0-4: RESULTS before during after before during after time interval (relative to msg. ] time interval (relative to msg. } CONCLUSIONS
  16. 16. INTRODUCTION METHOD RESULTS CONCLUSIONS 2. Distribution of Attention Gaze trajectories ; I as: Y (#175: ‘:1 1: “ I" »g= i I I 11.. {-3-’2[‘i-‘ E £5» ,1»
  17. 17. INTRODUCTION METHOD RESULTS CONCLUSIONS 2. Distribution of Attention Gaze trajectories 3-fit ‘: g=ET', "~$}" en-7.-s: —:a ; .;, «—s. :.i: : -* %_L_. Q<'_‘_t I; _.(-E4 »t. ,-$g~_': :', _.; >h*'_‘: —v_; ',t5»_'/ _ _ Y
  18. 18. 2. Distribution of Attention Gaze trajectories before instruction : KI 7,, during instruction INTRODUCTION METHOD RESULTS CONCLUSIONS
  19. 19. INTRODUCTION METHOD RESULTS CONCLUSIONS 2. Distribution of Attention Gaze trajectories before instruction during Instruction after instruction in _; _‘n in; a La» -< . L5,; :g. ;,. .‘, ‘t: .“M . bait. ; rri -+1‘; -fl:
  20. 20. 2. Distribution of Attention General visual processing pattern - Before instruction - Coarse scene overview - Several brief fixations, mainly foveating vehicles - Fixations connected by long-range saccades - During instruction | N‘| 'RQDUCT| QN - Focus of attention centers on areas verbally referred to. i. e. guided by navigation instruction - Some attention spent on relevant road furniture and route information METHOD - Little attention is being paid to other vehicles - After instruction RESU LTS - Focus of attention shifts back to vehicles‘ paths and relevant route areas for action planning CONCLUSIONS
  21. 21. 2. Distribution of Attention Special visual processing patterns - Inefficient visual processing - Complex scene. message issued early. verb at start, e. g. “Turn left at second junction. " - Saccades to first junction immediately after ''left” was issued - Distraction caused by redundancies - "Turn left into minor road alter bridge" when only one turn left exists INTRODUCTION - Sequential mapping of message constituents onto visual reference points leads to processing of redundant information instead of other task-relevant areas METHOD - Most critical in complex scenes - However. redundancies may be used as “back-up“ in simple scenes RESULTS CONCLUSIONS
  22. 22. 3. 3< cl. Incremental Generation of Representations Model: Mapping gaze trajectories onto navigation instructions - Procedure - Coding instructions and gaze trajectories in symbol sequences - Symbols represent words and fixations - Fixation locations determine fixation symbols (according to stimulus area) - Vehicle areas change dynamically - Comparison of symbol sequences indicates how (scene) representations INTRODUCTION ‘"3 9"'"e’a‘e“ METHOD "Tum left at second junction" He’ Hi ‘sf’ ‘as’ RESULTS CONCLUSIONS
  23. 23. 3. 8< cl. Incremental Generation of Representations Model: Mapping gaze trajectories onto navigation instructions - Findings - In general. the fixation sequence closely follows the instruction sequence - The visual focus is thus guided by the navigation instruction, very similar to the incremental generation of representations from spoken language in static scenes - However. depending on scene and message parameters. the incremental generation can be interrupted. Guidance of visual attention by instructions pauses INTRODUCTION when relevant route when other vehicles approach. information is attended to‘ .4 ,4 METHOD Y Instruction processing is resumed, skipping previous message pan 14. RESULTS F ‘I Most likely for complex Most likely for complex scenes and messages scenes and late messages CONCLUSIONS
  24. 24. Summary and Conclusions Analysis of eye movements revealed how visuo-linguistic interaction affected visual attention Distribution of attention changes considerably between the different phases of driving while following navigation instructions When a navigation instruction is issued, the visual focus is largely guided by the message However, relevant scene information and vehicle movements are also taken INTRODUCTION Into account Scene and message complexity and message timing affect the processing f h’ " I‘ f ' METHOD 0 t Is additiona In ormation If the sequential processing of information directly referred to in the instruction is interrupted for too long, the resumption of message RESULTS processing is critical and leads to increased error rates for message comprehension CONCLUSIONS
  25. 25. Summary and Conclusions Perception and processing of navigation messages are greatly compromised when visual attention moves away from items currently referred to by the instruction. suggesting a strong visuo-linguistic interaction between message perception (auditory). message processing (linguistic) and referencing (visual) as well as message comprehension (cognitive) It appears that verbal and visual information cannot be processed in parallel when an asynchrony exists between the current verbal input and the focus of visual attention INTRODUCTION The attempt to synchronise the auditory and visual input streams often fails as message comprehension deteriorates In the dynamic, time critical environments tested in the study METHOD It could thus be beneficial for navigation systems to dynamically generate (or adapt) its verbal messages depending on the current traffic situation RESULTS Navigation instruction generation guided by scene content could optimise the processing of concurrent tasks and thus improve driving safety CONCLUSIONS
  26. 26. Thank you for your attention! Hendrik Koesling & Honan 6 Reilly I he Interaction of Navigation Instructions and Visual Attention In Dynamic Automotive Environments NUI Maynooth. Ireland 9-ma/ I: hendrI'k@cs. nuIm. Ie

×