Design and Assessment of the Haptic Creature’s Affect Display. Presented at HRI '11: The 6th ACM/IEEE International Conference on Human-Robot Interaction. Lausanne, Switzerland; March 6-9 2011. More info at http://yohanan.org/steve/projects/haptic-creature/.

1. Design and Assessment of
the Haptic Creature’s Affect Display
Steve Yohanan and Karon E. MacLean
SPIN Research Group • Department of Computer Science • University of British Columbia

2. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 2
The Haptic Creature Project
Social HRI through Affective Touch
● Emotion communication is
an important aspect of social
interaction.
● Predominately displayed
through facial expression or
speech prosody.
● Touch is an under-explored
modality for affect display.
Emphasis on Companionship & Therapy

3. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 3
The Haptic Creature Project
Social HRI through Affective Touch
● Small, fur covered robot the
size of large cat or small dog.
● Emotion Expression
– Ears
– Breathing
– Vibrotactile Purr
● Emotion Sensing
– Touch Sensors
– Accelerometer
The Haptic Creature

4. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 4
The Haptic Creature Project
Research Approach
Affect Display from Robot

5. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 5
The Haptic Creature Project
Research Approach
Affect Display from Human

6. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 6
The Haptic Creature Project
Research Approach
Influence of Affect Display
Full Cycle of Interaction

7. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 7
Research Questions
● In what manner might the Haptic Creature communicate its
emotional state through touch?
● How effective is the Haptic Creature at communicating its
emotional state through touch?
● Does gender or experience with animals affect perception of
the Haptic Creature's emotional state?
Affect Display from Robot

8. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 8
Highlights
● Describe the Haptic Creature's affect display design.
– Arousal: ear stiffness & breathing rate.
– Valence: breath symmetry & purring.
● Present a hybrid technique developed for assessing
effectiveness of affect display.
● Found robot communicated arousal more clearly than valence.
● Provide suggestions for improvement.

9. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 9
Overview
➢ Related Robots
● Affect Display Design
● Assessment of Affect Display
● Results
● Suggested Improvements

10. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 10
Related Robots
● Intersection of ...
– socially interactive
robotics,
– zoomorphic,
– touch interaction, and
– affect display.
● Haptic Creature focuses on
affect display solely through
touch.
● Haptic Creature does not
represent a specific animal.
ShibataShibata StiehlStiehl
UgobeUgobe SonySony
Saldien & GorisSaldien & Goris

11. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 11
Overview
● Related Robots
➢ Affect Display Design
● Assessment of Affect Display
● Results
● Suggested Improvements

12. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 12
Affect Display Design
Affect Space
Valence and Arousal
Adapted from
Russell (1980, 1989)

13. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 13
Affect Display Design
Affect Space
Emotional State => (v,a)
Current Emotion

14. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 14
Affect Display Design
Affect Space
Behavior Specified by Key Expressions
Current Emotion
Key Expression
Behavior Interpolation

15. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 15
The Haptic Creature
Affect Display Parameters
Behavior Development
Behavior
Refinement
Behavior
Refinement
Behavior
Refinement
Animal
Models
Informal
Pilots
Mini-
Study
Informal
Pilots

16. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 16
The Haptic Creature
Affect Display Parameters
Breathing Rate
Ear Stiffness
Arousal
Breathing Symmetry
Valence
Purr Presence
Valence (Positive)
Purr Amplitude
Arousal

17. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 17
Overview
● Related Robots
● Affect Display Design
➢ Assessment of Affect Display
● Results
● Suggested Improvements

18. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 18
Assessment of Affect Display
Study Overview
Effectiveness of Communicating Emotional State
● Haptic Creature expressed a
variety of emotional states.
● Participants rated perceived
emotional state in two ways.
● Emotion Label
– Pinpoint specific emotion.
● Valence and Arousal
– Fine grain dimensional.
● These two measures provide
confirmation of each other.

19. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 19
Assessment of Affect Display
Response Format
● Forced choice from 16
emotion labels.
● 9 dimensional emotion labels
(Russell).
● 6 (*) distinct emotion labels
(Ekman).
● “None Of These” avoids
artificial agreement
(Frank and Stennett).
Afraid* Miserable
Angry* Neutral
Aroused Pleased
Depressed Relaxed
Disgusted* Sad*
Distressed Sleepy
Excited Surprised*
Happy* None Of These
Emotion Label

20. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 20
Assessment of Affect Display
Response Format
Perceived Valence and Arousal
Sad ↔ Happy
Calm ↔ Excited
Self-Assessment Manikin (Lang 1980)
7-Level Pictorial Scales

21. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 21
Assessment of Affect Display
● 32 Participants
(50% Female)
● Ages 19 to 50
(M 27.5, SD 9.37)
● 9 Unique Stimuli
x 2 Repetitions Per Block
x 3 Blocks
= 54 Total Stimuli

22. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 22
Assessment of Affect Display
Emotion Stimuli
Min, Max, Avg for Valence and Arousal
Stimulus

23. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 23
Overview
● Related Robots
● Affect Display Design
● Assessment of Affect Display
➢ Results
● Suggested Improvements

24. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 24
Results
Emotion Labeling
Matching Distinct with Dimensional Labels
● Russell's (9) emotion labels
map directly to affect space.
● Need method to map
Ekman's (6) emotion labels.
Distressed Aroused Excited
Miserable Neutral Pleased
Depressed Sleepy Relaxed

25. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 25
Results
Emotion Labeling
Matching Distinct with Dimensional Labels
● Valence and arousal ratings
were analyzed for each label
choice.
● Labels where both the
valence and arousal did not
statistically differ were
considered equivalent.
● “Surprised” not matched.
Distressed Aroused Excited
Miserable Neutral Pleased
Depressed Sleepy Relaxed
Happy
Sad
Afraid
Angry
Disgusted

26. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 26
Results
Emotion Labeling
● Robot correctly displayed its
affect state (by participant):
17% to 52%
(M 30%, SD 10%)
● No influence of gender or
experience with animals.
● “Distressed” dominated high
arousal conditions.
“Sleepy” dominated low
arousal.
Recognition Scoring
Aroused Excited
Miserable
Depressed Relaxed
Distressed Distressed
Distressed
Sleepy Sleepy
Neutral
Neutral
Pleased
Pleased
Sleepy
Sleepy
Distressed
Distressed

27. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 27
Results
Perceived Valence and Arousal
Arousal Expressed More Clearly Than Valence
Valence
Unpleasant Neutral Pleasant
Arousal
NeutralActivated Deactivated

28. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 28
Overview
● Related Robots
● Affect Display Design
● Assessment of Affect Display
● Results
➢ Suggested Improvements

29. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 29
Suggested Improvements
Breathing
● Breathing Rate successfully
conveyed arousal.
● Participant debriefing found
cue both for arousal and
valence.
– Faster breathing implying
negative valence.
Breathing Rate
Arousal &
Valence

30. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 30
Suggested Improvements
Breathing
● Breath Symmetry failed to
convey valence.
● Participant debriefing found
Breathing Depth as cue for
valence.
– Shallower depth implying
negative valence.
Breathing Depth
Valence

31. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 31
Suggested Improvements
Purring
Purr Presence
Valence
(Positive & Negative)
Purr Amplitude
Arousal
● Successfully portrayed
positive valence for
“Pleased”.
● “Excited” confused with
“Distressed”.
● High Amplitude implied
negative valence (shivering).
– Reduce intensity for
“Excited”.
● Leverage shivering concept
for negative valence purr
(e.g., “Miserable”).

32. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 32
In Summary
● Described the Haptic Creature's affect display design.
● Presented a hybrid technique for assessing effectiveness of
affect display.
● Found robot communicated arousal more clearly than valence.
– Stiffness (ears) successfully conveyed arousal.
Rate of Touch (breathing) successfully conveyed arousal.
– Vibration (purr) successfully conveyed positive valence.
Vibration Amplitude (purr) miscategorized valence.
– Symmetry of Touch (breathing) failed to convey valence.
● Gender and experience with animals did not influence
perception of the robot's emotional state.

33. Design and Assessment of the Haptic Creature’s Affect Display - Steve Yohanan - HRI 2011 33
Questions
Merci!