Presentation by Tyron Louw at http://icttp2016.com www.its.leeds.ac.uk/people/t.louw

1. Were they in the loop during automated driving?
Links between visual attention and collisions
11 August 2016 ICTTP 2016, Brisbane, Australia1
Tyron Louw
@tyronlouw
researchgate.net/profile/Tyron_Louw
Institute For Transport Studies
University Of Leeds, UK

2. Acknowledgements
11 August 2016 ICTTP 2016, Brisbane, Australia2
• CO-AUTHORS
• Natasha Merat
Oliver Carsten
Ruth Madigan
• AND COLLEAGUES AT ITS LEEDS:
Gustav Markkula
Anthony Horrobin
Michael Daly

3. Introduction
The first generation of partially-automated vehicles (SAE Level 2)
is already on our roads (SAE, 2014).
Drivers are still required to supervise the system and resume
control.
But what will they actually do?
11 August 2016 ICTTP 2016, Brisbane, Australia3

4. Possible Outcome 1 – Passive Fatigue
Prolonged monitoring takes drivers out of the loop (Merat et al.,
2014) inducing passive fatigue (Desmond & Hancock, 2001).
Reduces drivers’ attentional capacity (Desmond & Matthews, 1997)
and ability to detect, evaluate and respond to critical events.
Increasing the likelihood of crashes (Endsley, 1995; Hollnagel & Woods,
2005; de Winter et al., 2014).
11 August 2016 ICTTP 2016, Brisbane, Australia4

5. Possible Outcome 2 – Task Disengagement
Drivers may choose to engage in non-driving related activities
(Carsten et al., 2012).
Distracts from the supposed primary task of monitoring the
vehicle.
11 August 2016 ICTTP 2016, Brisbane, Australia5

6. Consequence of outcomes
Passive fatigue and task disengagement may hamper drivers’
ability to safely resume control from an automated driving
system (Neubauer et al., 2012; Merat et al., 2012)
Sudden changes to the road environment capture drivers’
attention, resulting in reduced visual scanning of the scene and
increased fixations towards changes (Chapman & Underwood, 1997;
Velichkovsky et al., 2002) which link directly to an increase in crashes
(Crundall, Shenton & Underwood, 2004).
11 August 2016 ICTTP 2016, Brisbane, Australia6

7. Experimental setup
Out of the loop (OOTL) manipulation
techniques
Altering drivers’ visibility of the road
scene during automation while they
completed visual and non-visual
tasks
Aim: varying drivers’ level of
awareness and engagement in the
driving task (Louw et al., 2015)
11 August 2016 ICTTP 2016, Brisbane, Australia7
NO FOG + n-back
e)

8. Manipulation Aims
No Fog: Control Condition
No Fog + n-back: Assess the effect of a cognitive task.
Light Fog: Simulate a process whereby limited visual
attention was directed towards the screen.
Heavy Fog: Simulate situations where the driver is looking
completely away from the road and is unaware
of the traffic conditions.
Heavy Fog + Quiz: Assess the effect of a visual task without a
physical distraction.
11 August 2016 ICTTP 2016, Brisbane, Australia8

9. Research questions
(i) How do the OOTL manipulations affect the location of
drivers’ first fixation after the uncertainty event
(ii) How are drivers’ fixations distributed over time
(iii) What is the relationship between fixations during the
uncertainty alerts and crash frequency.
11 August 2016 ICTTP 2016, Brisbane, Australia9

10. Participants
11 August 2016 ICTTP 2016, Brisbane, Australia10
• 75 drivers (41 male)
• 21-69 years (M=36.16, SD=12.38)
• Normal or corrected-to-normal vision
• Average annual mileage of 8290.46 (SD=6723.08)
• Years holding driving licence (M=16.22, SD=12.92)

11. University of Leeds Driving Simulator
11 August 2016 ICTTP 2016, Brisbane, Australia11
Jaguar S-type cab in a
4m spherical projection
dome
300° field-of-view
projection system
v4.5 Seeing Machines
faceLAB eye-tracker
recorded eye
movements at 60Hz

12. Drive Design
11 August 2016 ICTTP 2016, Brisbane, Australia12
No Fog Heavy FogLight Fog Heavy Fog + Task
Lead
vehicle
a Automation On
b Screen Manipulations On
c Drone Moves Into Lane
d Screen Manipulations Off/Uncertainty Alert
e Lead Vehicle Action
Non-critical Critical
1 2 3 4 5 6
≈150s
a b d ec
Ego
vehicle
No Fog + NBack
100 s 3 s 3 s
Not a takeover request,
but a request to monitor
and intervene if deemed
necessary

13. Areas of Interest
Centre: 6° circular region of the road centre area
Fixations: 200ms threshold was used with a standard deviation of gaze position below 1o
11 August 2016 ICTTP 2016, Brisbane, Australia13
Visual attention regions (Carsten et al., 2012)
Carsten O, Lai FCH, Barnard Y, Jamson AH, Merat N. Control task substitution in semi-automated driving: Does it matter what aspects are automated? Hum Factors 2012;54,747–761.

14. Where do drivers first look after the manipulations end?
11 August 2016 ICTTP 2016, Brisbane, Australia14
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
No Fog Light Fog Heavy Fog Heavy Fog +
Quiz
No Fog + n-back
Index of Dispersion
Percent of fixations
Centre Top Left Bottom Right Index of Dispersion
Each bar: Percent of total
number of fixations (First
fixation of 15 drivers for 6
events = 90 fixations) in
each Area of Interest for
one group.
Index of dispersion(id):
Calculation of how evenly
fixations are distributed
among the areas of interest

15. Looking at the Road Center in the first 3 s
6 X 3 X 5 mixed ANOVA
Event Number (1-6) and
Time (1-3) as within
Screen Manipulation (5
Conditions) between
11 August 2016 ICTTP 2016, Brisbane, Australia15
No effect of Screen Manipulation but interaction of Time*Screen
Manipulation = 1st Second differences
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1 s 2 s 3 s
Percent Road Center
**
**
Effect of Time
** p < .001

16. Looking at the Road Center in the first 3 s
11 August 2016 ICTTP 2016, Brisbane, Australia16
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Event 1 Event 2 Event 3 Event 4 Event 5 Event 6
Percent Road Center
Event 1 Event 2
**
*
**
Non-critical Critical
Effect of Event Number
* p < .01, ** p < .001

17. Fixations during the transition
11 August 2016 ICTTP 2016, Brisbane, Australia17
No Fog Heavy FogLight Fog Heavy Fog + Task
Lead
vehicle
a Automation On
b Screen Manipulations On
c Drone Moves Into Lane
d Screen Manipulations Off/Uncertainty Alert
e Lead Vehicle Action
Non-critical Critical
1 2 3 4 5 6
≈150s
a b d ec
Ego
vehicle
No Fog + NBack
100 s 3 s 3 s Crash
ü
💥
No Crash
VS
6 X 2 mixed ANOVA
Time Window (1s-6s after manipulations ended)
as within factor
Crash Outcome (collision/no collision) between
factor

18. Crash Outcomes
11 August 2016 ICTTP 2016, Brisbane, Australia18
Critical Event 1 Critical Event 2
No Crash Crash No Crash Crash
No Fog (N=15) 10 5 15 0
Light Fog (N=15) 14 1 14 1
Heavy Fog (N=15) 8 7 12 3
Heavy Fog + Quiz (N=15) 13 2 15 0
No Fog + n-back (N=15) 11 4 15 0
Total 56 19 71 4
Focus on Critical Event 1

19. Fixations based on Crash Outcome in Critical Event 1
11 August 2016 ICTTP 2016, Brisbane, Australia19
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
110%
120%
1 s before 1 s 2 s 3 s 4 s 5 s 6 s
PercentRoadCenter
No Collision in Critical Event 1 Collision in Critical Event 1 No Collision in Critical Event 2
**
*
SCREEN MANIPULATIONS END BRAKE LIGHT ONSET
**
* p<.05, ** p<.005
ü No Crash (N=54)
3 s after brake light
3 s after manipulations
💥 Crash (N=19)
Effect of Time Window
No effect of Crash Outcome
Interaction of Crash
Outcome*Time Window
No Difference
1s before

20. Summary
What drivers did during automation influenced where they
first looked, but differences disappeared within 2 s
Drivers’ visual attention allocation strategies adapted to
the uncertainty alert
11 August 2016 ICTTP 2016, Brisbane, Australia20

21. Summary
Non-colliders identified hazard early and had consistent
scanning pattern
Colliders late to identify hazard and experienced
perceptual tunnelling
11 August 2016 ICTTP 2016, Brisbane, Australia21

22. Conclusion (1)
Drivers actively seek out hazards and anticipate their
response
This is demanding in manual, but more so in automation.
11 August 2016 ICTTP 2016, Brisbane, Australia22

23. Conclusion (2)
With increasing automation, drivers’ decisions will shift
from being about when to act to whether to act
Implication
Drivers need to possess a confident understanding of their role
and the capabilities of their highly automated driving systems
11 August 2016 ICTTP 2016, Brisbane, Australia23

24. Conclusion (3)
The type of the hazard has a strong influence on what is
fixated and how quickly it is processed
Implication
Highly automated driving systems should discriminate hazards
leading to automation disengagement, and direct to drivers’
visual attention to them in sufficient time to respond
appropriately
11 August 2016 ICTTP 2016, Brisbane, Australia24

25. Thanks!
t.l.louw@leeds.ac.uk
For more information look out for:
11 August 2016 ICTTP 2016, Brisbane, Australia25