This document discusses research on the automated detection of frustration. It defines frustration as an emotional response to obstacles that can negatively impact learning. It describes how machines can be trained to detect frustration through physiological measurements, facial expressions, and performance on tasks like the Stroop test. Researchers are developing adaptive tutoring systems that can detect learner frustration and adjust teaching strategies using these automated detection techniques to improve learning outcomes.

1. AUTOMATED
DETECTION OF
FRUSTRATION

2. What is Frustration?
▪ Emotional response to obstacle
▪ External and internal obstacles
▪ Goal commitment and self-efficacy
(Lazar, Jones, and Shneiderman 240; Latham and Locke 218–219)

3. Impact of Frustration on Learning
▪ Flow and confusion
▪ Confusion  Frustration/boredom  Avoidance/abandonment
▪ Effort avoidance (defensive vs. intelligent)
▪ Frustration negatively correlated with learning
(Lehman, Matthews, D’Mello, and Person 51; Snow, Corno, and Jackson
275)

4. Automated Detection
▪ Assessment made by machine (computer algorithm)
▪ Artificial intelligence, machine learning, pattern recognition
▪ Initial training by humans
▪ After training, computers equally competent
(Barreto, Zhai, and Adjouadi 29; D'Mello, Picard, and Graesser 53–60)

5. Affective Computing
▪ Subfield of human-computer interaction studies
▪ Adaptive response to user state
▪ Improvement of user experience
▪ Feedback/assistance
(Barreto, Zhai, and Adjouadi 29; Puri et al. 1725)

6. AutoTutor
▪ Artificially intelligent tutoring system
▪ Adapts teaching strategy to learner’s progress
▪ Inclusion of emotional component
▪ Goal: superior learning outcomes
(D'Mello, Picard, and Graesser 53–60)

7. Physiological Measurements
▪ Heart rate, galvanic skin response, skin temperature, pupil diameter, …
▪ Reflect “fight or flight” response
▪ Multiple-modality sensors
▪ Sensor fusion improves classification accuracy
(Barreto, Zhai, and Adjouadi 29–32; Puri et al. 1725)

8. Stroop Color Word Conflict Test
▪ YELLOW
▪ Available time decreases
▪ Correlation with energy metabolism
(Barreto, Zhai, and Adjouadi 29–38; Puri et al. 1726–1728)

9. Non-invasive Technologies
▪ Infrared camera: thermal signature of face
▪ Web cam: facial expressions
(Puri et al. 1726–1728; Grafsgaard et al. 159–165)

10. Works Cited
Barreto, Armando, Jing Zhai, and Malek Adjouadi. “Non-intrusive Physiological Monitoring for Automated
Stress Detection in Human-Computer Interaction.” Human-Computer Interaction, edited by Michael
Lew, Nicu Sebe, Thomas S. Huang, and Erwin M. Bakker, Springer, 2007, 29–38.
D'Mello, Sidney, Rosalind Picard, and Arthur Graesser. “Toward an Affect-Sensitive AutoTutor.” IEEE Intelligent
Systems, vol. 22, no. 4, 2007, 53–61.
Grafsgaard, Joseph F., Joseph B. Wiggins, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester.
“Automatically Recognizing Facial Indicators of Frustration: A Learning-Centric Analysis.” Humaine
Association Conference on Affective Computing and Intelligent Interaction (ACII), 2013, 159–165.
Latham, Gary P. and Edwin A. Locke, “Self-Regulation through Goal Setting.” Organizational Behavior and
Human Decision Processes, vol. 50, 1991, 212–247.
Lazar, Jonathan, Adam Jones, and Ben Shneiderman. “Workplace User Frustration with Computers: An
Exploratory Investigation of the Causes and Severity.” Behaviour & Information Technology, vol. 25,
no. 3, 2006, 239–251.
Lehman, Blair, Melanie Matthews, Sidney D’Mello, and Natalie Person. “What Are You Feeling? Investigating
Student Affective States During Expert Human Tutoring Sessions.” Intelligent Tutoring Systems,
edited by Beverley P. Woolf, Esma Aïmeur, Roger Nkambou, and Susanne Lajoie, ITS ’08, Springer,
2008, 50–59.
Puri, Colin, Leslie Olson, Ionannis Pavlidis, James Levine, and Justin Starrn. “StressCam: Non-contact
Measurement of Users' Emotional States through Thermal Imaging.” CHI '05 Extended Abstracts on
Human Factors in Computing Systems, 2005, 1725-1728.
Snow, Richard E., Lyn Corno, and Douglas Jackson III. “Individual Differences in Affective and Conative
Functions.” Handbook of Educational Psychology, edited by David C. Berliner and Robert C. Calfee,