Di ss2013 hesita_presentation_final

The 6th Workshop on Disfluency in Spontaneous Speech
Stockholm, Sweden August 21-23, 2013
Sara Candeias 1
Dirce Celorico 1
Jorge Proença 1
Arlindo Veiga 1,2
Fernando Perdigão 1,2
1Instituto de Telecomunicações, Coimbra, Portugal
2University of Coimbra, DEEC, Portugal
HESITA(tions) in Portuguese
Database

2
DiSS 2013
Stockholm, Sweden - August 21-23, 2013
 Scope
 Goal
 Description of the HESITA Database
 Hesitation Patterns
 Hesitations across speaking styles
 Phonetic form of filled pauses
 Segmentation of hesitations
 Technical Information
 Future
SUMMARY

3
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

4
DiSS 2013
 LINGUISTIC or CLINICAL/THERAPEUTIC areas
 more directly interested in gathering knowledge for better
identifying salient information in human speech
communication
Various scientific domains can beneficiate of the analysis of the
hesitation distribution along the speech:
SCOPE

5
DiSS 2013
 LINGUISTIC or CLINICAL/THERAPEUTIC areas
 more directly interested in gathering knowledge for better
identifying salient information in human speech
communication
Various scientific domains can beneficiate of the analysis of the
hesitation distribution along the speech:
SCOPE
 SPEECH TECHNOLOGY
 to increase the usability of speech systems, by overpassing
the challenges proposed by the presence of such
phenomena.

6
DiSS 2013
AUTOMATIC LANGUAGE PROCESSING
 could benefit from a richer representation of the audio signal that
incorporates speaking styles information (hesitations),
 to reduce errors in the automatic speech recognition,
 to improve automatic conversational speech systems.
DETECTION OF HESITATION EVENTS
 provides the segmentation of multimedia data into consistent
parts,
 leads to important applications : identification of the speech
segments to train acoustic models for speech recognition in
spontaneous speech.
SCOPE

7
DiSS 2013
SCOPE
No database of hesitation events for
European Portuguese is freely
available so far !

8
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

9
DiSS 2013
Database for European Portuguese,
 mainly focused on the hesitation events,
 containing a large and rich variety of speech data events.
GOAL
HESITA database
Available through:
 Meta-Net: http://metanet4u.l2f.inesc-id.pt/repository/search/
 Project page: http://lsi.co.it.pt/spl/hesitation/downloads.html

10
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

11
DiSS 2013
HESITA Database
 30 daily news programs
 collected from podcasts of a European Portuguese television channel
 ~ 27 hours of speech
 audio downsampled from 44.1 kHz to 16 kHz sampling rate,
 video information discarded,
 studio and out of studio recordings, some telephone sessions.
DESCRIPTION OF THE HESITA DATABASE

12
DiSS 2013
HESITA Database
 prepared (read) speaking style is dominant:
 most of the speech encompasses utterances of anchors and
professional speakers (14 hours),
 spontaneous speech segments present:
 in commentators, reporters, interviewers and interviewees (10
hours),
 Lombard speech appears with low representativeness (18 minutes).

13
DiSS 2013
HESITA Database
Manually identified and annotated hesitation events:
patterns closely following the notation presented in E. Shriberg

14
DiSS 2013
HESITA Database
Manually identified and annotated hesitation events:
 repetitions (r),
 substitutions (s),
 filler words (p),
 deletions (d) and
 insertions (i).
 Only the speech segments were annotated in terms of hesitations,
 Filled pause vocalizations were transcribed using the SAMPA phonetic
alphabet for European Portuguese.

15
DiSS 2013
HESITA Database
Annotation encompasses information regarding to:
 audio characteristics - background environments:
 studio, street, speech overlapping, noise and music,
 acoustic events - non-speech events:
 music, jingles, laughter, coughing or clapping.
 respiratory and other events:
 noise from cars or wind,
 speaking style and speaker information.

16
DiSS 2013
HESITA Database
All the annotations were performed by using the Transcriber software tool.

17
DiSS 2013
SP_STU_E3_M represents:
 an annotation of speech segment (SP),
 in a noise-free environment, studio, (STU),
 with high level of spontaneity (E3), and
 from a male speaker (M).

18
DiSS 2013
SP_OVR_E3_M represents:
 the annotation of an speech segment (SP)
 with overlapping speech (OVR),
 in a spontaneous speaking style with high level of
spontaneity (E3), and
 from a male speaker (M).

19
DiSS 2013
 (r.r) - repetitions (r),
 (.w+) - extensions within a word (w+)
 (f.) - filled pauses (f).
 [6~]: (f.) - phonetic symbols attest extended vowel
sounds or vocalic fillers.
 res - presence of a respiratory event.

20
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

21
DiSS 2013
Considering the segments annotated accordingly to the presence of hesitations,
we can see how the hesitation patterns are distributed.
HESITATION PATTERNS
Top 10 most frequent hesitation patterns.
 total of 4608 events observed,
 filled pauses (f.) and vocalic extensions within
a word (.w+) are the most common.

22
DiSS 2013
Pattern models display the way that the hesitation occurs, indicating the order of
the words before and after the so-called “repair-point”.
HESITATION PATTERNS
Top 10 most frequent hesitation patterns.
 The repair point (as in (f.)) marks the place from
which the hesitation is repaired and the fluency
is restored.
 pattern (r.r) indicates that a word r was
repeated as repair or reinforcement ("de.de");
 pattern (s-.s), the word s was cut and then
substituted ("qua-.quantas");
 in (r2.r) the same word r was repeated twice
and finally restored ("com.com.com");
 in (rs-.rs) the word r was repeated and word s
was cut and, then substituted with correction
("da tu-.da totalidade").

23
DiSS 2013
More complex hesitation patterns are present…
HESITATION PATTERNS
Embedded hesitations:
"que vo-.que.que.que voltam.que.que possam"
" that re-.that. that. that return. that. that could "
( ( r s- .(r 2 . r) s) . ( r . r ) s).

24
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

25
DiSS 2013
In general hesitation events occur mainly in spontaneous speech
 4406 against 188 in read (prepared) speech and 12 in Lombard speech,
 total of 188 hesitations observed in 14 hours for read (prepared) speaking
style results in a rate of 0.22 hesitations per minute,
 4406 hesitation events in 10 hours of spontaneous speech result in a rate of
7.34 hesitations per minute.
HESITATIONS ACROSS SPEAKING STYLES
The density of hesitations in speech varies with the speaking style

26
DiSS 2013
Distribution of the 5 most common hesitation patterns in the read (prepared) speech
 High frequency of vocalic extensions (.w+) (39.36%)
just followed by filled pauses (f.) (32.45%).
Top 5 most frequent hesitation
patterns for read (prepared) speech.
Although the difference between those two occurrences is
not so expressive, it is possible that the choice for the
extensions reflects the fact that vocalic fillers tend to be
more stigmatized in a prepared speech context.

27
DiSS 2013
Distribution of the 5 most common hesitation patterns in the read (prepared) speech
 High frequency of vocalic extensions (.w+) (39.36%)
just followed by filled pauses (f.) (32.45%).
Top 5 most frequent hesitation
patterns for read (prepared) speech.
 Repetitions in read or prepared speech become
residual.
 The occurrence of substitutions are higher in the
prepared speech than in spontaneous speech (9.57%
vs. 3.61%).
„proving‟ that they are more adequate for communicative
strategy mainly in what the fluency of speaking is
concerned.

28
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

29
DiSS 2013
PHONETIC FORM OF FILLED PAUSES
 The two most common phonetic forms for
filled pauses:
 the near-open central vowel [ɐ] ([6] in
SAMPA),
 the mid-central vowel [ə] ([@] in SAMPA).
Phone distribution of filled pauses
(top10 most frequent).
This distribution supports the view that the
vocalizations preferred by Portuguese speakers
are around central vowels, corresponding to the
reduced vowels in an unstressed position.

30
DiSS 2013
filled pauses:
SAMPA),
 Slight
] as well (around 3%).
 A nasal preference is also evident : see [ɐ], [ɐm]
and [ m ].
Our point here is not to associate a meaning to the filler sounds. However, there is
strong empirical evidence that speakers use all of them for playing a structuring
role in the speech.

31
DiSS 2013
filled pauses:
SAMPA),
 Slight
] as well (around 3%).
 A nasal preference is also evident : see [ɐ], [ɐm]
and [ m ].
The choice for a vocalic sound rather than other appears to be, at least in some
contexts, motivated by the behavior of neighbor phonetic segments, neutralizing in
some way the phonetic difference of the vocalic fillers.

32
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

33
DiSS 2013
 Annotation of patterns closely follows E. Shriberg methodology,
 Encompasses the initial and final temporal marks,
 Corresponding label contains the pattern and the orthographic transcription,
 Repair-point marked temporally, showing the instant where the hesitation is
corrected and when the fluency on speech is recovered.
SEGMENTATION OF HESITATIONS
The period of time that corresponds to the beginning of the hesitation to its repair-
point is much larger (0.61 seconds in average) than the period of time between the
repair point and the end of the hesitation correction (0.34 seconds in average).
These trends concerning the distribution and duration of hesitation events may be
analyzed as manifestations of planning effort as well.

34
DiSS 2013
 Scope
 Goal
 Future
SUMMARY

35
DiSS 2013
Directories and files:
The archive that can be uploaded contains 58 audio files and the
corresponding TRS files, that enclose the two parts of the 30 daily
newsprograms.
Data structure of an entry:
The TRS files have a data type definition file associated: trans-14.dtd that
is provided in the archive.
TECHNICAL INFORMATION
Corpora size:
TRS files have a total of 4608 hesitation events.
The whole resource occupies 3GB, mainly due to the audio files.

36
DiSS 2013
Thank You
FUTURE...
We really expect that this database can be a relevant base of work for
further studies regarding a variety of speech phenomena.

The 6th Workshop on Disfluency in Spontaneous Speech
Stockholm, Sweden August 21-23, 2013
Sara Candeias 1
(saracandeias@co.it.pt)
Dirce Celorico 1
Jorge Proença 1
Arlindo Veiga 1,2
Fernando Perdigão 1,2
1Instituto de Telecomunicações, Coimbra, Portugal
2University of Coimbra, DEEC, Portugal
HESITA(tions) in Portuguese
Database

Di ss2013 hesita_presentation_final

Recommended

Recommended

More Related Content

Similar to Di ss2013 hesita_presentation_final

Similar to Di ss2013 hesita_presentation_final (15)

Recently uploaded

Recently uploaded (20)

Di ss2013 hesita_presentation_final

Editor's Notes