The use of eye movement in the study of emotionally expressive agent in multimedia environments

1. The use of eye movement in the study of emotionally
expressive agent in multimedia environments
Enilda Romero-Hall, Ph.D. (@RomeroEnilda)
Assistant Professor
Instructional Design & Technology
University of Tampa

2. Purpose
To explain the theoretical and empirical studies which support the
need for research on emotionally expressive animated agents
using visual attention measures.
2

3. What are animated agents?
3

4. Benefits of Animated Agents
¡ Positive impact on learners’ motivation and how strongly they
value computer-based learning programs
¡ Help learners focus on important elements of learning materials
¡ Provide learners with context-specific learning strategies and
advice
4

5. Benefits of Animated Agents
¡ Agent can demonstrate physical tasks, such as operation and
repair of equipment
¡ Agents can serve as the learners’ attentional guides using the
most common and natural methods: gaze and deictic gesture
5

6. Literature Review
Some studies have found that under certain conditions animated
agents may distract learners, overload them cognitively, or even
suppress learning
(Baylor & Ryu, 2003; Clark & Choi, 2007; Craig et al., 2002;
Dirkin, Mishra, & Altermatt, 2005)
6

7. Literature Review
There are few studies that highlight the use of an animated agents
as a factor contributing to higher learning scores and/or a more
positive attitude towards the learning experience
(Arslan-Ari, 2010; Baylor, 2002; Clarebout & Elen, 2006;
Kim et al., 2006; Moreno, 2001; Moreno et al., 2000;
Ryokai, Vaucelle, & Cassell, 2003; Yung, 2009).
7

8. Literature Review
The results of the published empirical studies on animated agents
are varied and the findings are inconclusive
8

9. Emotion and Animated Agents
9

10. Emotion and Animated Agents
An entertaining and compelling animated agent must express
emotion (Rickel et al., 2002).
The purpose is not just to add animated agents for prettiness, but
to leverage learners’ natural tendencies to attribute humanness
to the animated agent.
10

11. Visual Attention Measures in
Animated Agent Research
11

12. Visual Attention Measures in
Animated Agent Research
¡ Eye tracking methodology can help empirically substantiate
theories of human-computer interactions that assume that
humans are inclined to treat computers as social partners when
interacting with them (Reeves & Nass, 1996)
¡ One study has investigated how the presence of an educational
animated agent in a computer-mediated environment affected
the learners’ visual attention (Louwerse et al., 2009)
12

13. Effects of Emotion in Visual
Attention
13

14. Effects of Emotion in Visual
Attention
¡ Visual attention might be directed to certain kinds of events
because of intrinsic meaning or salience (Hunt, 2007). Due to
their adaptive relevance, emotional stimuli are expected to be
detected quickly (Calvo et al., 2007).
¡ Research on the preferential visual processing of emotional
scenes competing for attention resources with neutral scenes
revealed that there was preferential processing of emotional
scenes (Calvo et al., 2007).
14

15. Conclusion
¡ Research on emotionally expressive agents, using eye tracking
can provide unique information about what is visually attended
to, in what order, and for how long
(van Gog & Scheiter, 2010).
¡ Eye tracking data may enhance computer-meditated
instruction by significantly improving their design based on
knowledge of how learners process certain materials
(Hyönä, 2010; Mayer, 2010; van Gog & Scheiter, 2010).
15

16. Questions or Comments?
Contact
Email: eromerohall@ut.edu
LinkedIn: https://www.linkedin.com/in/romeroenilda
Twitter: @RomeroEnilda
16