Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams (Jing Su)

Copyright © CeADAR 201319/5/2013 Copyright © CeADARNatural Language Processing Dublin1
TopicListener
Observing Key Topics from!
Multi-Channel Speech Audio
Streams
Jing Su, Oisín Boydell

Copyright © CeADAR3/8/2016 Natural Language Processing Dublin
•Speech audio often encapsulates huge volumes of
information, but it has been challenging to mine and
analyse using automated methods. !
•Automated speech-to-text transcription technologies
get significant improvements in both accuracy and
cost.!
•Topic modelling techniques identify key themes and
topics from textual corpora and may be extended to
speech transcriptions.
2
Introduction

•TopicListener delivers key topics from multi-channel
speech audio streams!
▪ It combines advanced topic modelling techniques with automatic
speech transcription to identify key themes and topics across large
volumes of recorded audio conversations!
▪ It provides a novel means to explore and visualise the correlation
and evolution of topics over time!
•TopicListener offers a solution for unsupervised topic
extraction on call centre conversation recordings!
•Issues from a single operator can be reported, but they need a tool
to explore popular issues with limited human labour
3
Introduction

Copyright © CeADAR3/8/2016 Natural Language Processing Dublin4
Transcriber
Topic
Model
Transcriber
Transcriber
Transcriber
1 Ranked
2 Topics
3 From
4 Multichannel
5 Audio
6 Streams
Introduction
Modelling topics from multi-channel call centre audio transcriptions

Topic Modelling
http://www.cs.columbia.edu/~blei/fogm/2014F/lectures/introduction-slides.pdf

•Latent Dirichlet Allocation (LDA)!
▪ LDA1 defines a Dirichlet distribution between a document and
multiple topics!
•Hierarchical Dirichlet Process (HDP)!
▪ HDP2 assigns a Dirichlet process for each group of data and the
Dirichlet processes for all groups share a base distribution!
▪ HDP features a two level Dirichlet process
7
Topic Models
1. McCallum, Andrew Kachites. "MALLET: A Machine Learning for Language Toolkit." http://mallet.cs.umass.edu. 2002.
2. G. Heinrich, “Infinite LDA - Implementing the HDP with minimum code complexity,” arbylon.net, Tech. Rep., 2011

•The demo is build on an open audio repository!
▪11,120 video documentaries are collected from the Youtube Al
Jazeera English channel!
▪Audio commentaries were transcribed by Google’s automated
speech transcription engine!
▪The intl. news channel covers the contents of politics, military,
economy, sports, education, etc.!
▪This dataset is split into 35 weekly sub-corpora
8
Dataset

•LDA model complexity is
evaluated by a topic model
stability test !
▪ Train a reference LDA model
with 100% documents in a
weekly corpus!
▪ Train a candidate LDA model
with 80% randomly selected
documents in the same corpus!
▪ Compare model similarity with
Hungarian agreement score!
•LDA prefers a simple model
(k=2)
9
Topic Model Selection
●
●
●
●
●
●
2 3 4 5 6 7
0.20.30.40.5
Model agreement for 35 Al Jazeera news channel weekly corpora
number of topics for LDA model
Hungarianagreementscore
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
Topic model agreement scores of LDA models in 35
weekly Al Jazeera transcription corpora

•Average Jaccard Similarity1 (AJ)!
▪ AJ is a top-weighted Jaccard index which covers ranking
information!
▪ AJ gets the average of the Jaccard scores between every pair of
subsets of top-ranked terms in two lists, for depth d
10
Topic Similarity Metrics
news
music
tennis
bird
net
bird
news
wiki
cat
yellow
1. D. Greene, D. O’Callaghan, and P. Cunningham, “How many topics? stability analysis for topic models”, ECML
PKDD 2014

AJ Ri
,Rj( )=
1
t
γ d
Ri
,Rj( )d=1
t
∑
where
γ d
Ri
,Rj( )=
Ri,d
∩Rj,d
Ri,d
∪Rj,d

•Hungarian agreement score1!
▪ Hungarian agreement score measures the similarity between two
sets Sx and Sy both containing k ranked lists!
▪ a k by k matrix M registers the similarity between topic Rxi and Ryj . !
▪ Hungarian agreement score is the mean of the best match
between Rxi and Ry
11
Topic Model Similarity
1. D. Greene, D. O’Callaghan, and P. Cunningham, “How many topics? stability analysis for topic models”, ECML
PKDD 2014

agree Sx
,Sy( )=
1
k
AJ Rxi
,π Rxi( )( )i=1
k
∑

•Multi-channel speech audios are transcribed and re-
ordered as time series sub-corpora Ci!
•An HDP model is trained over Ci
’ which has stopwords
filtered off!
•Topic similarity matrix is over any two adjacent topic
models!
•An interactive UI shows topic models over all channels
in sequential order!
•The system works in an incremental scheme
13
System Architecture

Transcriber
Transcriber
Transcriber
Transcriber
T2 T1
Sequential
Alignment
Sequential and Interactive Display
of Topic Models
Time-series
Subcorpora
C1 M1
M2
Topic Model
Mi
S1,2
Si-1,i
Topic Similarity
between
Adjacent Models
Filtered
Subcorpora
C’1
Audio
Streams
TopicListener system architecture

•A proper UI illustrates the contents in an intuitive way!
▪Topic models are aligned in time sequence!
▪Popularity of a topic is symbolised by the size of a block!
▪Different topics in the same model are coloured differently!
▪Similar topics in consecutive windows are connected!
▪Emerging topics and ending topics are easily identified!
▪Keywords of a topic is shown on the right side with mouse click
15
TopicListener UI

TopicListener UI
2
1 3
2
1
3
Topics captured in March and April 2014 from automatic transcripts of the Al Jazeera news channel

TopicListener UI
Topics captured in March and April 2014 from automatic transcripts of the Al Jazeera news channel
1
1
2
2 3 3

•Speech transcription accuracy
▪ The sample corpus is from Google’s automatically generated
news channel captions and is reliable with manual inspection.
•Topic model stability
▪ Would topic models be stable against textual noise from audio
transcriptions?
▪ LDA model stability over deletion, insertion and replacement
errors is tested1.
•Visualisation
▪ The TopicListener system was tested over automatic transcripts
of call centre recordings from the financial servicing industry and
the response was that the visualisation was a useful tool.
18
Evaluation
1. Jing Su, Oisín Boydell, Derek Greene, Gerard Lynch, Topic Stability over Noisy Sources http://arxiv.org/abs/1508.01067

0.25
0.50
0.75
0.1 0.2 0.3 0.4 0.5
Noise Level of Input Texts
Hungarianscore
numTopics
3
5
10
15
20
30
Hungarian agreement of Ref. and Deletion mixd texts on bbc corpus
0.25
0.50
0.75
0.1 0.2 0.3 0.4 0.5
Hungarianscore
numTopics
3
5
10
15
20
30
Hungarian agreement of Ref. and insertion mixed texts on bbc corpus
0.25
0.50
0.75
0.1 0.2 0.3 0.4 0.5
Hungarianscore
numTopics
3
5
10
15
20
30
Hungarian agreement of Ref. and metaphone mixed texts on bbc corpus
LDAmodelHungarianagreementscores
withlevelsofDeletion, Insertion1 and
Replacement1 errorsinthe bbc2 corpus
1. 0% to 50% random terms from a list of frequent
English words with 7726 entries
2.D. Greene, D. O’Callaghan, and P. Cunningham, “How
many topics? stability analysis for topic models”,
ECML PKDD 2014

•We focus on the challenge of topic modelling on
unsupervised audio stream monitoring, and propose a
robust topic modelling tool in solving this problem
•The importance of timing in data processing and
topic modelling is highlighted
•The sequential and interactive UI illustrates the
evolution of topics in an intuitive way
•The incremental working scheme addresses the big
data challenge
•More details on CeADAR website:
http://ceadar.ie/pages/topiclistener-observing-key-topics-from-multi-channel-audio-streams/
20
Conclusion

Questions?
21

Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams (Jing Su)

Recommended

Recommended

More Related Content

Similar to Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams (Jing Su)

Similar to Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams (Jing Su) (20)

More from Sebastian Ruder

More from Sebastian Ruder (20)

Recently uploaded

Recently uploaded (20)

Topic Listener - Observing Key Topics from Multi-Channel Speech Audio Streams (Jing Su)