The document summarizes research on how virtual agents can mimic human nonverbal gestures and behaviors to build rapport with users. 1) Studies found that when users couldn't detect it, virtual agents that mimicked users' nonverbal gestures were perceived as more socially present and persuasive. 2) However, if users detected the mimicry, they found the agents less trustworthy and friendly. 3) Other research showed that when virtual agents were designed to be helpful to users (an "aid"), users felt more connected to agents that resembled them facially and had a more positive overall experience.

1. Bailenson, J. N., & Yee, N. (2005)
Digital Chameleons
Automatic Assimilation of Nonverbal Gestures in Immersive Virtual Environments
Eun Joung Cho

2. Look at these pictures.

4. And these . . .

6. Were you smiling?
Did you feel sad?

7. Motor mimicry
People change their own face similar to their interactant.
e.g. Our face is distorted when the other is in pain.
Chameleon effect
Mimicked behaviours lead to interactants’ favor.

8. Chameleon effect
Synchronization
• Accents
• likable
• Speech patterns
• prosocial behaviour
• Syntax
• rapport persists
• General mood

9. A salad,
B main dish, and A salad,
C dessert B main dish, and
…thank C dessert, please
you, ma’am.
… thank
you, ma’am.
(van Baaren, Holland, Steenaert, & van Knippenberg, 2003)

10. Because the information is all digital
1. Frequency (how many times)
2. Thoroughness (how many types of gestures)
3. Intensity (exact mirror or only an approximation)
Using mimicry in Virtual world
1. Strategic conversation with others using avatar.
2. Sociable computer system.

11. 2005 Digital Chameleons
2008 Detecting digital chameleons
2010 Effects of Facial Similarity on User Responses to Embodied Agents
Experiment video clip

12. 2005 Digital Chameleons
IV
• Participant’s gender
• Agent’s gender
• Agent’s behaviour (mimic or recorded)
DV
• agent’s social presence (realistic)
• agreement (agent’s persuasion)
• impression of the agent (positive)

13. 1.
2. Over 28˚, participants cannot see the avatar.
• Female : mimic -> do not
• Male : recorded -> do

14. 2008 Detecting digital chameleons
IV
Agent’s behaviour
• Mirror-mimic (exactly mirrored)
• Congruent-mimic (reverse-mirrored)
• Axis-switch (mirrored along a different axis)
DV
• trustworthiness
• warmth
• information
• agreement

15. • Participants rated the presenter as
less trustworthy when they detected the mimic.
less friendly (warmth)
• Participants were more likely
to detect the mirror-mimic condition than others.

16. 2010 Effects of Facial Similarity on User Responses to Embodied Agents
IV
• Designed similarity
(facial similar vs. dissimilar)
• Designed affordance
(aid vs. obstacle)
DV
• Involvement (How much do you feel connected to X?)
• Distance (How much does X leave you with cold feelings?)
• UseIntention (How much do you want to use X again?)

17. • Similar – dissimilar : no difference
• Aid – obstacle : difference
On average participants feel
more using intention with an aiding agent that was facially similar.
higher involvement with an aiding agent.
less distant with an aiding agent.

18. To sum up,
Mimicked behaviour  more social presence / better impression
But,
If the users notice the mimicry, it does not work
And,
The users feel positive when just the agents are helpful.

19. Thank you
Eun Joung Cho