Dr. Alan Pope's presentation at EngageUX during the World Usability Event organized by Tech3Lab at HEC Montréal on Nov 12th 2015.

1. mesures
d’engagement
Dr. Alan Pope
NASA
UX&

2. UX & Measures of Engagement
World Usability Day @ HEC:
User Engagement and Usability
Montreal, Canada
November 2015
Alan Pope
NASA Langley Research Center
1

3. 2
• Background of Crew State Monitoring Research
at NASA Langley
• Development of a Method for Evaluating
Task Engagement Indices
• In Progress Crew State Monitoring Research
at NASA Langley

4. 3
Real-Time State Estimation in a Flight Simulator Using fNIRS
Gateau T, Durantin G, Lancelot F, Scannella S, Dehais F (2015)
PLoS ONE 10(3): e0121279

5. 4
Initial Research Agenda

6. 5
• Background of Crew State Monitoring Research
at NASA Langley
• Development of a Method for Evaluating
Task Engagement Indices
• In Progress Crew State Monitoring Research
at NASA Langley

7. • ~70% of incidents and accidents are
attributed to human error
Human Error in Aviation
Incidents and Accidents
• A primary goal of NASA’s Aeronautics
research focus is to improve the National
Airspace System which already has an
exceptionally high level of safety.
6

8. Use of Automation in Aviation and
Hazardous States of Awareness (HSAs)
• Automation plays a significant role in the
cockpit
– enables humans to perform beyond normal
abilities (longer shifts, improved control, etc.)
• But can lead to suboptimal psychological
states [Aviation Safety Reporting System (ASRS)
database]
– complacency
– boredom
– diminished alertness
– compromised vigilance
– lapsing attention
– preoccupation
– absorption 7

9. 8

10. 9

11. Brainwave Correlates of Hazardous States
In
Advanced Concepts Flight Simulator
10

12. 11
• Background of Crew State Monitoring Research
at NASA Langley
• Development of a Method for Evaluating
Task Engagement Indices
• In Progress Crew State Monitoring Research
at NASA Langley

13. 12
Task engagement has been defined in various ways.
• A subjective state defined by questionnaire factor analytic
loadings on energetic arousal (affect), task motivation, and
concentration (cognition). Matthews et al. (2012)
• A cognitive measure that increases “as a function of
increasing task demands.” Task demand in the Berka et al.
study was manipulated with a set of tasks that elicited the
targeted cognitive states. Berka et al. (2007)
• “a substantial amount of the variance” associated with task
engagement could be explained using psychophysiological
measures. Fairclough and Venables (2006)

14. 13
The closed-loop evaluation method implies a
particularly dynamic definition of task
engagement
• how closely brain response slews with changes
in task demand when brain response is fed back
to adjust task demand (automation/manual mix)
in real time.
• represents an “operationalization” of the task
engagement construct (Lilienfeld et al., 2015).

15. 14

16. 15

17. 16
Biological
Psychology

18. 17
EEG data could be used to determine player experience
across entire level designs.
Nacke et.al (2010)
Engagement index (Beta / (Alpha + Theta)) was capable of
differentiating high intensity game events (Player Death)
from general game play.
McMahan, Parberry and Parsons (2015)
Engagement Index (EI) did not show systematic vigilance decrement
but discriminated cued and uncued conditions near task end.
Kamzanova, Kustubayeva and Matthews (2014)
Three EEG band ratios, beta/(alpha+theta), beta/alpha,
and beta/theta were able to discriminate
daydreaming from attentive driving
beta/alpha band ratio outperformed the other two indices.
Zhao, G., Wu, C. and Ou, B (2013).

19. 18
Adapting Automation based upon
EEG Measures of Task Engagement

20. 19
Spin-Offs from Attention and Engagement Research

21. 20
Eastern Virginia Medical School
Videogame Neurofeedback Research

22. 21
• How does this work fit within the NASA
mission?
– “Congress declares that the general welfare of the
US requires that the unique competence of NASA in
science and engineering systems be directed to
assisFng in bioengineering research, development,
and demonstra4on programs designed to alleviate
and minimize the effects of disability.” The Na'onal
Aeronau'cs and Space Act, Sec. 20102. (f)

23. 22
ZONE
An embodiment of U.S. patent number 8628333
Biofeedback
Training
for
Optimal
Athletic
Performance

24. 23
EEG-based MindShift FPS
Trade names and trademarks are used in this presentation for identification only.
Their usage does not constitute an official endorsement,
either expressed or implied, by the National Aeronautics and Space Administration.

25. 24
• Background of Crew State Monitoring Research
at NASA Langley
• Development of a Method for Evaluating
Task Engagement Indices
• In Progress Crew State Monitoring Research
at NASA Langley

26. 25

27. 26
Fixed-Base Simulator Lab

28. 27
Attention-related Human Performance Limiting
States (AHPLS)
• AHPLS may reduce pilot aircraft state awareness, and
can be indicated by covert or physiological markers of
limited performance.
• Current NASA Crew State Monitoring (CSM) studies
assess the efficacy of using CSM technology as a means
of detecting adverse human performance states.

29. 28
• CAST research for safety enhancements to improve
commercial pilot Training for Attention Management to
address attention-related human performance
limitations observed in flight incidents
• Develop methods to detect and measure attention-
related
human performance limiting states (AHPLS):
o Channelized Attention
o Diverted Attention
o Startle / Surprise
o Confirmation Bias

30. 29
Results are to be presented:
Harrivel, Liles, Stephens, Ellis, Prinzel, Pope.
Psychophysiological sensing and state classification for
attention management in commercial aviation.
AIAA Sci Tech 2016, San Diego, 4 - 8 January 2016.

31. 30
LaRC Cockpit Motion Facility

32. 31
Next CSM Research Objective Is Analogous
to the Biofeedback Spin-Offs – “Spin-Back”
• Commercial Aviation Safety Team (CAST)
recommendation to study Training for Attention
Management
• “Training-based mitigations - self-diagnosis methods
for flight crew members to recognize and recover
from channelized attention, confirmation bias, startle/
surprise, and diverted attention”
• Analogous to training to recognize symptoms of
hypoxia

33. 32
For coverage of our core research:
Stephens, C. L., Scerbo, M. W., and Pope, A.T.
Adaptive Automation for Mitigation of Hazardous States of Awareness
Chapter 26 in The Handbook of Operator Fatigue
edited by Matthews, Desmond, Neubauer, and Hancock, Ashgate 2012.
For coverage of our biofeedback work:
Pope, A.T., Stephens, C.L., and Gilleade, K. M.
Biocybernetic Adaptation as Biofeedback Training Method
Chapter 5 in Advances in Physiological Computing
edited by Fairclough and Gilleade, Springer 2014.

34. 33