This study examined the perception of emotions through auditory, visual, and auditory-visual modes in individuals with cochlear implants, hearing aids, and normal hearing. It found that individuals with normal hearing performed best in emotion perception, while cochlear implant users did not demonstrate an advantage over hearing aid users. While auditory-visual perception was better than single modes for normal hearing individuals, hearing loss individuals could not benefit from additional auditory information in the combined mode. Visual information dominated the combined mode. Perception scores correlated with age, as older participants performed better.

1. Auditory, visual, and auditory-visual
perception of emotions by individuals with
cochlear implants, hearing AIDS, and normal
hearing
Most, T. and Aviner, C. (2009). Journal of deaf studies
and deaf education, 14, 449-464.

2. Introduction
• Spoken language communication comprises
linguistic information (Mozziconacci, 2002)
– lexical items
– syntactic pattern
– vocal nonverbal information
– speaker’s emotional state with respect to the topic or
the listener
• This study focuses upon nonverbal information,
specifically upon the emotional state of the
speaker.

3. Introduction
• Perception of the speaker’s emotional state
– auditory
• fundamental frequency, intensity, duration, rate of speech
• variations in acc of diff emotions
– easy: anger, sadness
– difficult: surprise, disgust
– visual
• eye fear, anger, sadness
• mouth happiness, disgust
– easy: happy
– difficult: fear, disgust
– between: anger, sadness

4. Introduction
• Compare perception of emotions through
different mode
– visual mode (V) > auditory mode (A)
– A+V > A
– A+V ?(> or =) V

5. Introduction
• Emotion Perception by Individuals With Hearing
Loss Using Hearing Aids

6. Introduction
• Hearing loss (HL)
– diff perceiving the spoken signal in general
– diff perceiving auditory nonverbal cues of emotions
• Auditory perception of emotions
– HAs (hearing aids) < NH (normal hearing)
– negative correlation (emotions and degree of HL)
• Visual perception of emotions
– HAs ?(>, =, <) NH

7. Introduction
• Emotion Perception by Individuals With Hearing
Loss Using Cochlear Implants

8. Introduction
• Increases the audibility of the speech signal
 better speech perception
– CI> HAs (with similar degree HL)
• Variables
– age of implantation
– duration of CI use
– onset of deafness

9. Purpose
• Evaluate emotions perception
– CI, HAs and NH
• Hypothesis
– auditory emotion perception: NH> all HL
– auditory emotion perception: CI> HA
– A and the A+V modes: early-CI> late-CI
– NH: A+V mode> V mode
– HL: A+V> A or V mode
– the perception of specific emotions through the
different modes by all the participants

10. Materials and Methods
• Participants:
Number of Age Duration (CI)
Group participants HL range (HA)
Early CI 10(4 10:10-15:7 6:9-13:1
males) (M= 3:11, SD= 1:3) (M= 9:7, SD= 2.0)
Late CI 10(4 10-17:6 1-6:9
males) (M= 15:1, SD= 2:7) (M= 3.8, SD=
2:10)
HA 10(5 10:2-15:9 73-110 dB
males) (M= 13:10, SD= (M= 88, SD= 9.6)
1.11)
NH 10(4 10:5-16:10
males) (M= 14:10, SD=

11. Materials and Methods
• Instrument:
– Identification of Emotion Test (IET)
– total of 36 video-recorded
• anger, fear, sadness, happiness, disgust, and surprise
• the same neutral sentence ‘‘I am going out now and I’ll be
back later.’’

12. Materials and Methods
• Procedure:
– three individual sessions
• emotion vocabulary test
• Ling 6-sound test
• emotion test- three modes (block design)
– group (4)*mode (3)*emotion (6)

13. Results
• Scores were adjusted according to the following
formula by boothroyd (1998) to take gussing inti
account:

14. Results
• Group results (group(4)*mode(3)) :
– main effect:
• group: F(3, 36)= 6.47, p< .01
• mode: F(2, 35)= 303.88, p< .001
– interaction: F(6, 70)= 5.26, p< .001

15. Results

16. Results
• Identification of Specific Emotions
(group(4)*mode(3)*emotion(6))
– main effect:
• mode effect for emotion
• group effect for emotion (happiness, sadness and disgust)
– interaction:
• between group and mode (happiness, anger and disgust)

17. Results
• A mode:
– happiness, sadness and disgust: NH better than other
groups
– anger: NH> LCI & HA, NH= ECI
– fear: NH> ECI & LCI, NH= HA

18. Results
• V mode:
– No sig diff between group and emotion.
– perceive order (all group):
• happiness> anger, sadness and disgust> surprise and fear

19. Results
• A+V mode:
– HA no sig diff from other group.
– surprise: NH> LCI, ECI
– perceive order (all group):
• happiness> anger> disgust> fear

20. Results
• V, A+V modes revealed a similar hierarchy on
emotion identification, but A mode different.
– happiness was the easiest to perceive when presented
in the V, A+V modes
– sadness was easier to perceive when presented in the
A modes than other modes

21. Results
• Pearson correlations (mode, group)
– sig: V and A+V scores for the ECI (r= .82, p< .01)
and LCI (r= .68, p< .05) and in the NH (r= .88, p< .
001)
• Pearson correlations (mode, age, duration, degree
of HL, age at onset of hearing loss)
– sig: chronological age and A+V scores (r= .45, p< .
01) and the V scores (r= .32, p< .05)

22. Discussion
• Hypothesis
– auditory emotion perception: NH> all HL (O)
– auditory emotion perception: CI > HA
– A and the A+V modes: early-CI > late-CI
– NH: A+V mode > V mode
– HL A+V > A or V mode
– the perception of specific emotions through the
different modes by all the participants

23. Discussion
• The auditory emotion perception of individuals
with NH would be superior to that of all
participants with hearing loss was supported.

24. Discussion
• Hypothesis
– auditory emotion perception: NH> all HL
– auditory emotion perception: CI > HA (X)
– A and the A+V modes: early-CI > late-CI
– NH: A+V mode > V mode
– HL A+V > A or V mode
– the perception of specific emotions through the
different modes by all the participants

25. Discussion
• The CI users did not demonstrate any advantage
over the HA users in identifying emotions
through either the auditory or the auditory–visual
mode.
– segmental features

26. Discussion
• Hypothesis
– auditory emotion perception: NH> all HL
– auditory emotion perception: CI > HA
– A and the A+V modes: early-CI > late-CI (X)
– NH: A+V mode > V mode
– HL A+V > A or V mode
– the perception of specific emotions through the
different modes by all the participants

27. Discussion
• No significant differences emerged between the
early and late implantees in any of the
presentation modes, and no significant correlation
emerged between the perception of emotions and
age at implantation.
– both small group
– age (2.6)

28. Discussion
• Hypothesis
– auditory emotion perception: NH> all HL.
– auditory emotion perception: CI > HA
– A and the A+V modes: early-CI > late-CI
– NH: A+V mode > V mode (O)
– HL A+V > A or V mode
– the perception of specific emotions through the
different modes by all the participants

29. Discussion
• NH auditory identification of emotions was
significantly lower than visual identification or
combined auditory–visual identification.
– utilize the auditory information and to benefit from
the additional auditory information received about
emotions in the combined mode.

30. Discussion
• Hypothesis
– auditory emotion perception: NH> all HL.
– CI users would perform differently from HA users in
auditory emotion perception
– A and the A+V modes: early-CI > late-CI
– NH: A+V mode > V mode
– HL: A+V > A or V mode (X)
– the perception of specific emotions through the
different modes by all the participants

31. Discussion
• HL were unable to benefit from the additional
auditory information provided in the combined
mode.
– Individual difference

32. Discussion
• Hypothesis
– auditory emotion perception: NH> all HL.
– CI users would perform differently from HA users in
auditory emotion perception
– A and the A+V modes: early-CI > late-CI
– NH: A+V mode > V mode
– HL A+V > A or V mode
– examine the perception of specific emotions through
the different modes by all the participants

33. Discussion
• Visual information was more dominant in the
combined mode.

34. Discussion
• Chronological age did correlate sig with
– visual
– auditory–visual modes
– older participants obtaining better perception scores
• Mental knowledge
– representation of the typical emotions
– grows with age

35. Discussion
• Future studies (emotion identity)
– age at implantation
– CI combine HA v.s. CI alone
– postlingually deafened v.s. prelingually deafened