1. Speech & Language
Technology
A.J.vanHessen@ewi.utwente.nl
Searching
in
spoken
words
Disclosure of recorded content in MediaMosa
SURFnet Relatiedagen 2010
Noordwijkerhout December 9, 2010

2. • Introduction
– Why
speech
is
so
important
– What
is
HLT?
• Working
applications:
– Self-­‐service
(Internet
Telephony)
– Searching
in
recorded
audiovisual
recordings
• Demonstrations

3. Humans as speaking creatures
The
start
of
the
human
speech
started
some
100.000
years
ago.
Before,
the
shape
of
the
vocal
track
was
not
“ready”
for
the
modern
speech.
The
larynx
was
situated
too
high,
something
you
can
see
with
chimps.

4. Humans as writing
creatures
Sumer
(3300
AD,
Mesopotamia)
is
probably
the
oldest
written
language.
NU
-­‐100.000
-­‐10.000
-­‐3300
speech
farming
text

5. What
is
HLT?
• Human
Language
Technology
is
the
technology
that
mimics
the
human
language
capacity.
speech text sign

6. Redundancy
• Vlgones
een
oznrdeeok
op
een
Eglnese
uvinretsiet
mkaat
het
neit
uitin
wlkee
vloogdre
de
ltteers
in
een
wrood
saatn,
het
einge
watblegnaijrk
is
is
dat
de
eretse
en
de
ltaatse
ltteer
op
de
jiutsepatals
saatn.
De
rset
van
de
ltteers
mgoen
wllikueirg
gpletaastwdoren
en
je
knut
vrelvogens
gwoeon
lzeen
wat
er
saatt.
Dit
kmotodmat
we
neit
ekle
ltteer
op
zcih
lzeen
maar
het
wrood
als
gheeel.

7. Pensez a
ceque v
s avez f
nit et dema
n dez vs i
est coq
ue 3 ai

8. Dialogue
systems
(telephony,
real
time,
limited
complexity)
Disclosure
systems
(high
quality
audio,
ofQline,
complex)
WORKING
APPLICATIONS

9. Spoken
Document
ContactCenter
Retrieval Information
Retrieval
Voice
HLT
Natural
Language
Search
Web
Mens-­‐Machine-­‐
Emotie
detectie:
Mobile
Dictation Lachen/Huilen Communicatie

10. Companies
using
speech
technology

11. How
may
I
help
you
Why are they
Who is calling? calling?
Classification based on
Identification via ZIP-
the recognition of the
code and house number
question: “how may I
help you”

12. OrganisaCons
using
speech
technology

13. Disclosure
of
audiovisual
archives
• The
number
of
AV-­‐archives
on
the
Internet
increases
rapidly
• Archiving
is
not
enough:
disclosure
and
reusing
is
required!
• The
use
of
HLT
is
needed
(humans
cost
too
much).

14. Digitalized (historic) LVSR
Digital recorded collections
collections
WFH
H.M. Koningin
Second feministic wave Wilhelmina
Buchenwald
Memories of Indonesia

15. Searching
in
historic
radio
recordings:
Radio
Oranje

16. Oral
History:
Buchenwald

17. Oral
History:
Brandgrens,
RoKerdam
10
getuigen
van
het
bombardement
van
Rotterdam
(mei
‘40)
vertellen
hun
verhaal.
TST
wordt
gebruikt
om
in
de
getuigenissen
te
zoeken.

18. Searching
in
the
radio
interviews
of
WFH

19. Searching
in
46
interview
collecCons:
getuigenverhalen
(600
hour)

20. Searching
in
500
interviews
in
CroaCa

21. CroMe
-­‐
Audio
Search
Searching
for:
commandant
Phrase
boundaries
Found
word
5
fragments
(5x
commandant)
found

22. CroMe
-­‐
Audio
Search
Search
word
traumas
Language
found

23. PoliCcal
meCngs

24. Parliament
transcriptions
Gisteren
was
er
een
bespreking
ivm
de
betrekkingen
tussen
Nederland
en
Vlaanderen

25. RecogniCon
of
lectures
• Record
the
speech
• Record
the
PPT
• Recognise
the
speech
• Use
the
display
time
of
each
slide
as
THE
time
unit
• Use
the
recognised
speech
as
keywords
for
each
slide

26. Searching
in
news
broadcasts

27. Metadata
-­‐
Language
model
Text in the slide(s)
Lectures handouts
Language model
Environmental texts

28. QuesCons?