SlideShare a Scribd company logo
1 of 1
Download to read offline
Exploring Convolutional and Recurrent Neural Networks
in Sequential Labelling for Dialogue Topic Tracking
Seokhwan Kim, Rafael E. Banchs, Haizhou Li
Human Language Technology Department, Institute for Infocomm Research (I2
R), Singapore
Dialogue Topic Tracking
Categorizing the topic state at each time step
f(t) =



B-{c ∈ C} if ut is at the beginning of a segment belongs to c,
I-{c ∈ C} else if ut is inside a segment belongs to c,
O otherwise,
Examples of dialogue topic tracking
Speaker Utterance Topic
Guide How can I help you? B - OPEN
Tourist Can you recommend some good places to visit in Singa-
pore?
B - ATTR
Guide Well if you like to visit an icon of Singapore, Merlion will be a
nice place to visit.
I - ATTR
Tourist Okay. But I’m particularly interested in amusement parks. B - ATTR
Guide Then, what about Universal Studio? I - ATTR
Tourist Good! How can I get there from Orchard Road by public
transportation?
B - TRSP
Guide You can take the red line train from Orchard and transfer
to the purple line at Dhoby Ghaut. Then, you could reach
HarbourFront where Sentosa Express departs.
I - TRSP
Tourist How long does it take in total? I - TRSP
Guide It’ll take around half an hour. I - TRSP
Tourist Alright. I - TRSP
Guide Or, you can use the shuttle bus service from the hotels in
Orchard, which is free of charge.
B - TRSP
Tourist Great! That would be definitely better. I - TRSP
Guide After visiting the park, you can enjoy some seafoods at the
riverside on the way back.
B - FOOD
Tourist What food do you have any recommendations to try there? I - FOOD
Guide If you like spicy foods, you must try chilli crab which is one of
our favourite dishes.
I - FOOD
Tourist Great! I’ll try that. I - FOOD
Model 1: Convolutional Neural Networks (CNNs)
Convolutional neural network architecture for dialogue topic tracking
ut-1
ut
ut-2
ut-h+1
…
Input utterances
within window size h
Embedding layer with three different channels
for current, previous, and history utterances
Convolutional layer
with multiple kernel sizes
Max pooling
layer
Dense layer
w softmax output
Representing an utterance as a matrix with n rows of k-dimensional word vectors
Each input has three channels for the current, previous, and the history utterances
A convolutional filter has the same width k and a window size m as its height
The maximum value is selected from each feature map in max pooling layer
The values from max pooling are forwarded to the fully-connected softmax layer
Model 2: Recurrent Neural Networks (RNNs)
Recurrent neural network architecture for dialogue topic tracking
ut-h+1
…
ut-2
ut-1
ut
Inputs Utterance-level
embedding layer
sf
t-h+1
sf
t-2
sf
t-1
sf
t
Forward
layer
sb
t-h+1
sb
t-2
sb
t-1
sb
t
Backward
layer
yt-h+1
…
yt-2
yt-1
yt
Output
labels
Each utterance is represented with k-dimensional pre-trained embeddings
A sequence of the utterance vectors within h time steps are connected
Hidden states from uni-/bi-directional recurrent layers are passed to softmax
Model 3: Recurrent Convolutional Networks (RCNNs)
Recurrent convolutional network architecture for dialogue topic tracking
…
Inputs
…
ut-1
ut
ut-2
ut-h+1
Convolutional
layer
Forward
layer
sf
t-1
sf
t
sf
t-2
sf
t-h+1
Backward
layer
sb
t-1
sb
t
sb
t-2
sb
t-h+1
Output
labels
yt-1
yt
yt-2
yt-h+1
Max pooling
layer
Each feature vector generated after the max pooling layers in the CNN architecture
is connected to the recurrent layers in the RNN architecture
Evaluation
TourSG corpus
Human-human mixed initiative dialogues
35 sessions, 21 hours, 31,034 utterances
Manually annotated with nine topic categories
Models
Baselines
Support Vector Machines (SVM)
Conditional Random Fields (CRF)
CNNs: learned from scratch/pre-trained word2vec
RNNs: uni-directional/bi-directional RNNs/LSTMs
RCNNs: uni-directional/bi-directional RCNNs/LRCNs
Results
Models Features P R F
SVM bag-of-ngrams, speaker 59.85 59.94 59.90
SVM doc2vec, speaker 46.66 52.31 49.32
SVM bag-of-ngrams, speaker, doc2vec 59.91 60.01 59.96
CRF bag-of-ngrams, speaker 60.05 60.97 60.51
CRF doc2vec, speaker 61.77 49.57 55.00
CRF bag-of-ngrams, speaker, doc2vec 60.08 61.00 60.54
CNN learned from scratch 63.88 62.87 63.37
CNN learned from pre-trained word2vec 66.91 68.61 67.75
RNN uni-directional 49.51 53.75 51.55
RNN bi-directional 48.73 49.82 49.27
LSTM uni-directional 49.45 50.23 49.84
LSTM bi-directional 48.42 48.77 48.59
RCNN uni-directional 67.08 68.67 67.86
RCNN bi-directional 67.25 69.39 68.30
LRCN uni-directional 67.50 69.04 68.26
LRCN bi-directional 67.60 69.62 68.59
Error Distributions
SVM CRF CNN LRCN
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
5500
6000
6500
7000
7500
Numberoferrors
missing
extraneous
wrong category
wrong boundary
1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore 138632 Email: kims@i2r.a-star.edu.sg

More Related Content

What's hot

Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...
Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...
Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...Voica Gavrilut
 
Hash array mapped trie
Hash array mapped trieHash array mapped trie
Hash array mapped trieParidVaroshi1
 
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...Seokhwan Kim
 
Cs6503 theory of computation april may 2017
Cs6503 theory of computation april may 2017Cs6503 theory of computation april may 2017
Cs6503 theory of computation april may 2017appasami
 
11 - 03 Feb - From Recursion to Dynamic Programming
11 - 03 Feb - From Recursion to Dynamic Programming11 - 03 Feb - From Recursion to Dynamic Programming
11 - 03 Feb - From Recursion to Dynamic ProgrammingNeeldhara Misra
 
TBS 604 Mobile Computing.doc
TBS 604 Mobile Computing.docTBS 604 Mobile Computing.doc
TBS 604 Mobile Computing.docBhupesh Rawat
 

What's hot (7)

Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...
Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...
Scheduling in Time-Sensitive Networks (TSN) for Mixed-Criticality Industrial ...
 
Hash array mapped trie
Hash array mapped trieHash array mapped trie
Hash array mapped trie
 
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
 
Cs6503 theory of computation april may 2017
Cs6503 theory of computation april may 2017Cs6503 theory of computation april may 2017
Cs6503 theory of computation april may 2017
 
11 - 03 Feb - From Recursion to Dynamic Programming
11 - 03 Feb - From Recursion to Dynamic Programming11 - 03 Feb - From Recursion to Dynamic Programming
11 - 03 Feb - From Recursion to Dynamic Programming
 
Model toc
Model tocModel toc
Model toc
 
TBS 604 Mobile Computing.doc
TBS 604 Mobile Computing.docTBS 604 Mobile Computing.doc
TBS 604 Mobile Computing.doc
 

Similar to Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling for Dialogue Topic Tracking

Cheatsheet convolutional-neural-networks
Cheatsheet convolutional-neural-networksCheatsheet convolutional-neural-networks
Cheatsheet convolutional-neural-networksSteve Nouri
 
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...Seokhwan Kim
 
Dmcpresentation 120904112322 phpapp01
Dmcpresentation 120904112322 phpapp01Dmcpresentation 120904112322 phpapp01
Dmcpresentation 120904112322 phpapp01sridevi jalakam
 
Digital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCET
Digital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCETDigital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCET
Digital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCETSeshaVidhyaS
 
Intelligent Handwriting Recognition_MIL_presentation_v3_final
Intelligent Handwriting Recognition_MIL_presentation_v3_finalIntelligent Handwriting Recognition_MIL_presentation_v3_final
Intelligent Handwriting Recognition_MIL_presentation_v3_finalSuhas Pillai
 
Deep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.pptDeep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.pptShankerRajendiran2
 
Deep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.pptDeep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.pptSatyaNarayana594629
 
Deep learning for detection hate speech.ppt
Deep learning for detection hate speech.pptDeep learning for detection hate speech.ppt
Deep learning for detection hate speech.pptusmanshoukat28
 
12337673 deep learning RNN RNN DL ML sa.ppt
12337673 deep learning RNN RNN DL ML sa.ppt12337673 deep learning RNN RNN DL ML sa.ppt
12337673 deep learning RNN RNN DL ML sa.pptManiMaran230751
 
Deep-Learning-2017-Lecture ML DL RNN.ppt
Deep-Learning-2017-Lecture  ML DL RNN.pptDeep-Learning-2017-Lecture  ML DL RNN.ppt
Deep-Learning-2017-Lecture ML DL RNN.pptManiMaran230751
 
Crypto theory practice
Crypto theory practiceCrypto theory practice
Crypto theory practiceYoung Alista
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practiceHarry Potter
 
Crypto theory practice
Crypto theory practiceCrypto theory practice
Crypto theory practiceFraboni Ec
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practiceLuis Goldster
 
Crypto theory practice
Crypto theory practiceCrypto theory practice
Crypto theory practiceJames Wong
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practiceTony Nguyen
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practiceDavid Hoen
 

Similar to Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling for Dialogue Topic Tracking (20)

Cheatsheet convolutional-neural-networks
Cheatsheet convolutional-neural-networksCheatsheet convolutional-neural-networks
Cheatsheet convolutional-neural-networks
 
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
 
Dmcpresentation 120904112322 phpapp01
Dmcpresentation 120904112322 phpapp01Dmcpresentation 120904112322 phpapp01
Dmcpresentation 120904112322 phpapp01
 
Unit IV_SS_MMS.ppt
Unit IV_SS_MMS.pptUnit IV_SS_MMS.ppt
Unit IV_SS_MMS.ppt
 
Digital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCET
Digital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCETDigital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCET
Digital electronics(EC8392) unit- 1-Sesha Vidhya S/ ASP/ECE/RMKCET
 
Lti system
Lti systemLti system
Lti system
 
Intelligent Handwriting Recognition_MIL_presentation_v3_final
Intelligent Handwriting Recognition_MIL_presentation_v3_finalIntelligent Handwriting Recognition_MIL_presentation_v3_final
Intelligent Handwriting Recognition_MIL_presentation_v3_final
 
Deep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.pptDeep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.ppt
 
Deep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.pptDeep-Learning-2017-Lecture6RNN.ppt
Deep-Learning-2017-Lecture6RNN.ppt
 
RNN.ppt
RNN.pptRNN.ppt
RNN.ppt
 
Deep learning for detection hate speech.ppt
Deep learning for detection hate speech.pptDeep learning for detection hate speech.ppt
Deep learning for detection hate speech.ppt
 
12337673 deep learning RNN RNN DL ML sa.ppt
12337673 deep learning RNN RNN DL ML sa.ppt12337673 deep learning RNN RNN DL ML sa.ppt
12337673 deep learning RNN RNN DL ML sa.ppt
 
Deep-Learning-2017-Lecture ML DL RNN.ppt
Deep-Learning-2017-Lecture  ML DL RNN.pptDeep-Learning-2017-Lecture  ML DL RNN.ppt
Deep-Learning-2017-Lecture ML DL RNN.ppt
 
Crypto theory practice
Crypto theory practiceCrypto theory practice
Crypto theory practice
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practice
 
Crypto theory practice
Crypto theory practiceCrypto theory practice
Crypto theory practice
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practice
 
Crypto theory practice
Crypto theory practiceCrypto theory practice
Crypto theory practice
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practice
 
Crypto theory to practice
Crypto theory to practiceCrypto theory to practice
Crypto theory to practice
 

More from Seokhwan Kim

The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)Seokhwan Kim
 
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Seokhwan Kim
 
The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)Seokhwan Kim
 
Natural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionNatural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionSeokhwan Kim
 
The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)Seokhwan Kim
 
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Seokhwan Kim
 
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Seokhwan Kim
 
Sequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog StatesSequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog StatesSeokhwan Kim
 
Wikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingWikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingSeokhwan Kim
 
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...Seokhwan Kim
 
MMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionMMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionSeokhwan Kim
 
A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...Seokhwan Kim
 
A spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessA spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessSeokhwan Kim
 
An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...Seokhwan Kim
 
An Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionAn Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionSeokhwan Kim
 
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템Seokhwan Kim
 
A Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionA Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionSeokhwan Kim
 
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...Seokhwan Kim
 

More from Seokhwan Kim (18)

The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)
 
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
 
The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)
 
Natural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionNatural Language in Human-Robot Interaction
Natural Language in Human-Robot Interaction
 
The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)
 
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
 
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
 
Sequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog StatesSequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog States
 
Wikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingWikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic Tracking
 
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
 
MMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionMMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognition
 
A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...
 
A spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessA spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information access
 
An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...
 
An Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionAn Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information Extraction
 
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
 
A Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionA Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation Detection
 
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
 

Recently uploaded

Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?Mark Billinghurst
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandIES VE
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuidePixlogix Infotech
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Patrick Viafore
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxFIDO Alliance
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?Paolo Missier
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfSrushith Repakula
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...FIDO Alliance
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...marcuskenyatta275
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 

Recently uploaded (20)

Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 

Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling for Dialogue Topic Tracking

  • 1. Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling for Dialogue Topic Tracking Seokhwan Kim, Rafael E. Banchs, Haizhou Li Human Language Technology Department, Institute for Infocomm Research (I2 R), Singapore Dialogue Topic Tracking Categorizing the topic state at each time step f(t) =    B-{c ∈ C} if ut is at the beginning of a segment belongs to c, I-{c ∈ C} else if ut is inside a segment belongs to c, O otherwise, Examples of dialogue topic tracking Speaker Utterance Topic Guide How can I help you? B - OPEN Tourist Can you recommend some good places to visit in Singa- pore? B - ATTR Guide Well if you like to visit an icon of Singapore, Merlion will be a nice place to visit. I - ATTR Tourist Okay. But I’m particularly interested in amusement parks. B - ATTR Guide Then, what about Universal Studio? I - ATTR Tourist Good! How can I get there from Orchard Road by public transportation? B - TRSP Guide You can take the red line train from Orchard and transfer to the purple line at Dhoby Ghaut. Then, you could reach HarbourFront where Sentosa Express departs. I - TRSP Tourist How long does it take in total? I - TRSP Guide It’ll take around half an hour. I - TRSP Tourist Alright. I - TRSP Guide Or, you can use the shuttle bus service from the hotels in Orchard, which is free of charge. B - TRSP Tourist Great! That would be definitely better. I - TRSP Guide After visiting the park, you can enjoy some seafoods at the riverside on the way back. B - FOOD Tourist What food do you have any recommendations to try there? I - FOOD Guide If you like spicy foods, you must try chilli crab which is one of our favourite dishes. I - FOOD Tourist Great! I’ll try that. I - FOOD Model 1: Convolutional Neural Networks (CNNs) Convolutional neural network architecture for dialogue topic tracking ut-1 ut ut-2 ut-h+1 … Input utterances within window size h Embedding layer with three different channels for current, previous, and history utterances Convolutional layer with multiple kernel sizes Max pooling layer Dense layer w softmax output Representing an utterance as a matrix with n rows of k-dimensional word vectors Each input has three channels for the current, previous, and the history utterances A convolutional filter has the same width k and a window size m as its height The maximum value is selected from each feature map in max pooling layer The values from max pooling are forwarded to the fully-connected softmax layer Model 2: Recurrent Neural Networks (RNNs) Recurrent neural network architecture for dialogue topic tracking ut-h+1 … ut-2 ut-1 ut Inputs Utterance-level embedding layer sf t-h+1 sf t-2 sf t-1 sf t Forward layer sb t-h+1 sb t-2 sb t-1 sb t Backward layer yt-h+1 … yt-2 yt-1 yt Output labels Each utterance is represented with k-dimensional pre-trained embeddings A sequence of the utterance vectors within h time steps are connected Hidden states from uni-/bi-directional recurrent layers are passed to softmax Model 3: Recurrent Convolutional Networks (RCNNs) Recurrent convolutional network architecture for dialogue topic tracking … Inputs … ut-1 ut ut-2 ut-h+1 Convolutional layer Forward layer sf t-1 sf t sf t-2 sf t-h+1 Backward layer sb t-1 sb t sb t-2 sb t-h+1 Output labels yt-1 yt yt-2 yt-h+1 Max pooling layer Each feature vector generated after the max pooling layers in the CNN architecture is connected to the recurrent layers in the RNN architecture Evaluation TourSG corpus Human-human mixed initiative dialogues 35 sessions, 21 hours, 31,034 utterances Manually annotated with nine topic categories Models Baselines Support Vector Machines (SVM) Conditional Random Fields (CRF) CNNs: learned from scratch/pre-trained word2vec RNNs: uni-directional/bi-directional RNNs/LSTMs RCNNs: uni-directional/bi-directional RCNNs/LRCNs Results Models Features P R F SVM bag-of-ngrams, speaker 59.85 59.94 59.90 SVM doc2vec, speaker 46.66 52.31 49.32 SVM bag-of-ngrams, speaker, doc2vec 59.91 60.01 59.96 CRF bag-of-ngrams, speaker 60.05 60.97 60.51 CRF doc2vec, speaker 61.77 49.57 55.00 CRF bag-of-ngrams, speaker, doc2vec 60.08 61.00 60.54 CNN learned from scratch 63.88 62.87 63.37 CNN learned from pre-trained word2vec 66.91 68.61 67.75 RNN uni-directional 49.51 53.75 51.55 RNN bi-directional 48.73 49.82 49.27 LSTM uni-directional 49.45 50.23 49.84 LSTM bi-directional 48.42 48.77 48.59 RCNN uni-directional 67.08 68.67 67.86 RCNN bi-directional 67.25 69.39 68.30 LRCN uni-directional 67.50 69.04 68.26 LRCN bi-directional 67.60 69.62 68.59 Error Distributions SVM CRF CNN LRCN 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 5500 6000 6500 7000 7500 Numberoferrors missing extraneous wrong category wrong boundary 1 Fusionopolis Way, #21-01 Connexis (South Tower), Singapore 138632 Email: kims@i2r.a-star.edu.sg