Nicole Karpinsky defended her thesis which examined how spatial memory load affects compliance with an imperfect automated decision aid during a visual detection task. The study found that while the aid improved performance, spatial memory load reduced compliance compared to a no load condition. Compliance was higher for a near-perfect aid than a more fallible aid. The results provide insights into how workload influences human-automation interaction, though have some differences from prior studies that examined the impact of distracted conditions on compliance. Addressing limitations and exploring attentional demands could help characterize automation use in attention-demanding environments.

1. Thesis Defense
Nicole D. Karpinsky
September 29, 2016
The effect of concurrent visuospatial
memory demand on automation use
in a visual detection task

2. Automation can reduce workload, improve efficiency, and
increase safety for human operators (https://google.com/)

3. Automated Aids and Human Performance
Sensory
Processing
Perception
Decision
Making
Response
Selection &
Execution
Information
Acquisition
Information
Analysis
Decision
Selection
Action
Implementation
Human-Information
Processing Model
(Wickens et al., 2015)
Stages/Levels of
Automation
(Parasuraman, Sheridan, & Wickens, 2000)

4. HIGH
LOW
10. The computer decides everything, acts autonomously, ignoring the human
9. Informs the human only if it, the computer, decides to
8. Informs the human only if asked, or
7. Executes automatically, then necessarily informs the human, and
6. allows the human a restricted time to veto before automatic execution, or
5. Executes that suggestion if the human approves, or
4. Suggests one alternative
3. Narrows the selection down to a few, or
2. The computer offers a complete set of decision/action alternatives, or
1. The computer offers no assistance: human must take all decisions and actions.
LOA of Decision and Action Selection
(Sheridan & Verplank, 1978)

5. Automation Use
• Appropriate automation usage
allows operator to allot tasks to
automation to increase safety
and performance (Lee, 2008; Lee & See, 2004)
• Unfortunately, operators do not
always use automation as
prescribed
• misuse and disuse (Parasuraman & Riley, 1997)

6. Multiple Resource Theory
(Wickens, 2002)

7. Yamani-Horrey Attentional Model
(Yamani & Horrey, under review)

8. Signal Detection Theory (SDT)
• Statistical theory of decision
making in uncertain situations
• Characterizes operators’
responses toward imperfect
automated decision aids
• Isolates sensitivity and
response bias towards these
decisions

9. Signal Detection Theory (SDT)
Sensitivity (d’)
• Measures ability to discriminate
a stimulus from noise
• Represented as distance
between the means of signal
and noise distributions
Response bias (c)
• Indicates amount of evidence
needed to accumulate to make
a response
• Yes-No responses

10. Imperfect Automation Systems
• Compliance: Degree to which the operator takes action when the
system indicates that there is a signal
(Meyer et al., 2004)
Compliance = C Control – C Alert

11. Automation Use and Workload
• Previous studies examined the effects of increased cognitive load
on automation use
• In an automated warehouse-management system control task,
operators’ compliance increased with higher levels of workload (McBride
et al., 2011)
• Competition for attentional resources did not interfere with
response choices toward imperfect automation, resulting in no
compliance (Botzer et al., 2013)

12. Research Question
How does spatial memory load modulate
operators’ compliance towards an imperfect
decision aid?

13. Hypothesis 1: Main Effect of Automation
• Operators will comply with responses issued by an automated
decision aid less when performance interference occurs between
primary (aided) and secondary (unaided) tasks than when it does
not

14. Hypothesis 2: Main Effect of Load
• Operators will comply with the aid less in the spatial memory
condition than the single task condition

15. Hypothesis 3: Interaction
• Operators will comply with the aid similarly in the non-spatial
memory and the single task conditions

16. Design
• 3 × 2 × 2 mixed design
• Memory Load: NVSM, VSM, ST
• Automation Availability: Present vs. Absent
• Automation Type: FA-prone (PPV = .75 and NPV = .95) vs.
Near Perfect (PPV = .95 and NPV = .95)

17. Participants
• 30 participants from ODU participated in the study
• 23 females, (M = 24.65 years, SD = 4.92 years)
• Participants were screened for normal to corrected-to-normal visual
acuity and normal color vision
• Received research credit through the SONA system

18. Apparatus
• 23.6” LED monitor (1920 × 1080 px)
oRefresh rate of 60 Hz
• E-Prime 2.0
• Computer mouse
• Dimmed and quiet room

19. Visual Search Task
• Distractor letters Q and target
letter O, positioned at 4
random angles
• Each display contained 25
letters in 5 × 5 pattern
• Target was present half of trials
• Position and orientation were
random each trial

20. Non-Visual Spatial
Memory Task
• Solid colored squares, positioned at 4 set
locations
• Color of squares were randomly selected
• Memory test probe display contained 1
colored square at the middle of the display
• Probe square contained 1 memory stimuli
half of the trials

21. Visual Spatial Memory
• Solid black squares, positioned at 8
possible locations
• Location of squares were randomly
selected
• Memory test probe display contained 1
empty square at 1 of the 8 locations
• Probe square appeared at 1 of the
locations of the memory stimuli half of the
trials

22. Procedure

23. Statistical Analysis
• All statistical analyses adopted standard level of alpha (.05)
• Data with incorrect responses in memory tasks were removed
• Log-linear transformations were used to correct for extreme STD
scores (Hautus, 1995)
• Greenhouse-Geisser used to correct any sphericity violations
• Bonferroni used for post-hoc analyses

24. Results: Sensitivity, d’
• Sensitivity was greater for aided than unaided conditions
• (M = 1.36 vs. .67), true for FA-Prone and NP conditions
• Sensitivity was greater for Spatial Load than No Load condition
• (M = 1.24 vs. .73)
• NP condition had greater sensitivity in Spatial Load and Non-Spatial
Load conditions

25. Results: Sensitivity, d’
• For the NP condition, sensitivity
was greater in Spatial Load
condition compared to No Load
and Non-Spatial Load condition
• Automation Availability was
more pronounced for Spatial
Load condition compared to No
Load condition

26. Results: Compliance, c
• Compliance was greater
when the aid was present in
No Load than Non-Spatial
Load condition
• Compliance was also
greater in FA-Prone than NP
system

27. Discussion
• As expected, the aid improved search performance in all conditions
• There was a cost of dual-tasking towards compliance
• Results are similar and different from previous research
• Phillips and Madhavan, 2011
• Karpinsky et al., 2016

28. Discussion
Phillips & Madhavan, 2011
• Similar: Automation improved
performance in the aided
condition compared to the
unaided conditions
• Different: Compliance was
higher in all distracted
conditions compared to the
non-distracted condition
Karpinsky et al., 2016
• Similar: Participants complied
less as performance
interference increased
• Different: Compliance was
higher in all distracted
conditions compared to the
non-distracted condition

29. Discussion
Limitations
• Memory performance was
lower in the current study
compared to previous research
(76.5% vs. 65%)
Future Considerations
• Researchers should look into
the effects of attentional
demand incurred by secondary
tasks on automation use
• Will help to further characterize
HAI in attention demanding
environments