SlideShare a Scribd company logo
1 of 1
Download to read offline
Dynamic Memory Networks for Dialogue Topic Tracking
Seokhwan Kim
Adobe Research, San Jose, CA, USA
Dialogue Topic Tracking
Categorizing the topic state at each time step
f(t) =



B-{c ∈ C} if ut is at the beginning of a segment belongs to c,
I-{c ∈ C} else if ut is inside a segment belongs to c,
O otherwise,
Examples of dialogue topic tracking
Speaker Utterance (ut) f(t)
Guide How can I help you? B-OPEN
Tourist Can you recommend some good places to visit in Singa-
pore?
B-ATTR
Guide Well if you like to visit an icon of Singapore, Merlion will
be a nice place to visit.
I-ATTR
Tourist Okay. But I’m particularly interested in amusement
parks.
B-ATTR
Guide Then, what about Universal Studio? I-ATTR
Tourist Good! How can I get there from Orchard Road by public
transportation?
B-TRSP
Guide You can take the red line train from Orchard and transfer
to the purple line at Dhoby Ghaut. Then, you could reach
HarbourFront where Sentosa Express departs.
I-TRSP
Tourist How long does it take in total? I-TRSP
Guide It’ll take around half an hour. I-TRSP
Tourist Alright. I-TRSP
Guide You could spend a whole afternoon at the park by its
closing time at 6pm.
B-ATTR
Tourist Sounds good! I-ATTR
Guide Then, I recommend you enjoy dinner at the riverside on
the way back.
B-FOOD
Tourist What do you recommend there? I-FOOD
Guide If you like spicy foods, you must try chili crab which is
one of our favorite dishes.
I-FOOD
Tourist Great! I’ll try that. I-FOOD
Baselines: CNN and RCNN (Kim et al., 2016)
CNN RCNN
Inputs
ut-1
ut
ut-w+1
…
ut-2ut-2
Convolutional
layer
Max pooling
layer
Prediction
yt
Inputs
ut-w+1
…
ut-1
ut
ut-2ut-2
Convolutional
layer
Max pooling
layer
Max pooling
layer
Recurrent
layer
ht-w+1
…
ht-2
ht-1
ht
Prediction
yt
Prediction
yt
Convolutional Neural Network (CNN) for Dialogue Topic Tracking
Representing an utterance as a matrix with n rows of k-dimensional word vectors
A convolutional filter has the same width k and a window size m as its height
The maximum value is selected from each feature map in max pooling layer
The values from max pooling are forwarded to the fully-connected softmax layer
Recurrent CNN (RCNN) for Dialogue Topic Tracking
Each feature vector generated after the max pooling layers in the CNN
architecture is connected to the recurrent layers in the RNN architecture
Proposed Model: Dynamic Memory Network
Dynamic Memory Network (DMN) for Dialogue Topic Tracking
Inputs
ut-w+1
…
ut-1
ut
ut-2ut-2
Convolutional
layer
Dynamic
Memories
Prediction
yt
Prediction
yt
h1
t-w+1 h2
t-w+1 hm
t-w+1…
h1
t-2 h2
t-2 hm
t-2…
h1
t-1 h2
t-1 hm
t-1…h1
t-1 h2
t-1 hm
t-1…
h1
t h2
t hm
t…h1
t h2
t hm
t…
…
…
…
Max
pooling
Proposed Model: Dynamic Memory Network
Our models represent the latent dialogue state at each given time
step as a set of read-writable memory slots
Each memory slot is updated through a given dialogue sequence by
the content-based operations in gated recurrent networks
Gating mechanisms
Single Gate (Henaff et al. 2016) Update and Reset Gates Cross-slot Interactions
zj
i σ uT
i wj
+ uT
i hj
i−1 σ k αkj
z uT
i wk
+ βkj
z uT
i hk
i−1
rj
i - σ uT
i Wr wj
+ uT
i Ur hj
i−1 σ k αkj
r uT
i wk
+ βkj
r uT
i hk
i−1
˜hj
i tanh Uhj
i−1 + Vwj
+ Wui tanh U rj
i ◦ hj
i−1 + Vwj
+ Wui
hj
i 1 − zj
i ◦ hj
i−1 + zj
i ◦ ˜hj
i
Evaluation: Data
TourSG corpus
Human-human mixed initiative dialogues
35 sessions, 21 hours, 31,034 utterances
Manually annotated with eight topic categories
‘attraction’, ‘transportation’, ‘food’, ‘accommodation’, ‘shopping’, ‘opening’, ‘closing’, ‘other’
15 classes: ({B-, I-} × {c : c ∈ C; and c = ‘other’}) ∪ {O}
Data Statistics
Set # sessions # segments # utterances
Train 14 2,104 12,759
Dev 6 700 4,812
Test 15 2,210 13,463
Total 35 5,014 31,034
Evaluation: Implementation Details
Word Embedding
Initialized with the pre-trained word2vec on 2.9M sentences from travel forum
Fine-tuned while the whole model is trained
Convolutional Layer
Learned 100 feature maps for each of three different filter sizes {3, 4, 5}
For CNN, applied over the current, previous, and history utterances w = 10
For RCNN and DMN, applied for each single utterance
Recurrent Layer
RCNNs
We compared two variants: Vanilla RNNs and Gated Recurrent Units (GRUs)
The hidden layer dimensions were 150 for the vanilla RNN and 50 for the GRU
DMNs
Three dynamic memory networks were trained based on the proposed gating mechanisms
The number of memory slots were m = 5 for the first two distributed models and m = 10 for
the other with cross-slot interactions
Model Training
Adam optimizer by minimizing the categorical cross entropy loss on softmax
With mini-batch size of 50 and dropout after max pooling with the rate of 0.25
Stopped training after 150 epochs where the CNN baseline has been saturated.
Evaluation: Results
Sequential Labelling Segmentation
Models P R F Pk WD
CNN 0.6691 0.6861 0.6775 0.3799 0.4884
RCNN (RNN) 0.6825 0.6572 0.6696 0.3970 0.4634
RCNN (GRU) 0.6936 0.6826 0.6880 0.3888 0.4619
DMN (single) 0.6877 0.7105 0.6989 0.3782 0.4393
DMN (reset & update) 0.6959 0.7035 0.6997 0.3781 0.4427
DMN (cross-slot) 0.7008 0.7090 0.7049† 0.3532‡ 0.4223‡
CNN RCNN Dynamic Memory
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Numberoferrors
missing
extraneous
wrong boundary
wrong category
Slot 0 Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Slot 8 Slot 9
B-ATTR
B-TRSP
B-FOOD
B-ACCO
B-SHOP
I-ATTR
I-TRSP
I-FOOD
I-ACCO
I-SHOP
0.6
0.7
0.8
0.9
345 Park Avenue, San Jose, CA 95110, USA Email: seokim@adobe.com

More Related Content

What's hot

Travelling Salesman Problem
Travelling Salesman ProblemTravelling Salesman Problem
Travelling Salesman ProblemDaniel Raditya
 
Towards a stable definition of Algorithmic Randomness
Towards a stable definition of Algorithmic RandomnessTowards a stable definition of Algorithmic Randomness
Towards a stable definition of Algorithmic RandomnessHector Zenil
 
Deep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for FinanceDeep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for FinanceBen Ball
 
Machine learning session 9
Machine learning session 9Machine learning session 9
Machine learning session 9NirsandhG
 
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...Hector Zenil
 
Fractal dimension versus Computational Complexity
Fractal dimension versus Computational ComplexityFractal dimension versus Computational Complexity
Fractal dimension versus Computational ComplexityHector Zenil
 
Computability - Tractable, Intractable and Non-computable Function
Computability - Tractable, Intractable and Non-computable FunctionComputability - Tractable, Intractable and Non-computable Function
Computability - Tractable, Intractable and Non-computable FunctionReggie Niccolo Santos
 
Advance compositing and animation
Advance compositing and animationAdvance compositing and animation
Advance compositing and animationpaiils111
 
論文輪読資料「Gated Feedback Recurrent Neural Networks」
論文輪読資料「Gated Feedback Recurrent Neural Networks」論文輪読資料「Gated Feedback Recurrent Neural Networks」
論文輪読資料「Gated Feedback Recurrent Neural Networks」kurotaki_weblab
 
Fuzzieee-98-final
Fuzzieee-98-finalFuzzieee-98-final
Fuzzieee-98-finalSumit Sen
 
Practical Reinforcement Learning with TensorFlow
Practical Reinforcement Learning with TensorFlowPractical Reinforcement Learning with TensorFlow
Practical Reinforcement Learning with TensorFlowIllia Polosukhin
 
Different techniques for speech recognition
Different  techniques for speech recognitionDifferent  techniques for speech recognition
Different techniques for speech recognitionyashi saxena
 
P, NP, NP-Complete, and NP-Hard
P, NP, NP-Complete, and NP-HardP, NP, NP-Complete, and NP-Hard
P, NP, NP-Complete, and NP-HardAnimesh Chaturvedi
 

What's hot (19)

Travelling Salesman Problem
Travelling Salesman ProblemTravelling Salesman Problem
Travelling Salesman Problem
 
Towards a stable definition of Algorithmic Randomness
Towards a stable definition of Algorithmic RandomnessTowards a stable definition of Algorithmic Randomness
Towards a stable definition of Algorithmic Randomness
 
Deep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for FinanceDeep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for Finance
 
Cn tutorial (6 cs119)
Cn tutorial (6 cs119)Cn tutorial (6 cs119)
Cn tutorial (6 cs119)
 
poster
posterposter
poster
 
Machine learning session 9
Machine learning session 9Machine learning session 9
Machine learning session 9
 
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
Fractal Dimension of Space-time Diagrams and the Runtime Complexity of Small ...
 
Internet
InternetInternet
Internet
 
Fractal dimension versus Computational Complexity
Fractal dimension versus Computational ComplexityFractal dimension versus Computational Complexity
Fractal dimension versus Computational Complexity
 
Computability - Tractable, Intractable and Non-computable Function
Computability - Tractable, Intractable and Non-computable FunctionComputability - Tractable, Intractable and Non-computable Function
Computability - Tractable, Intractable and Non-computable Function
 
Advance compositing and animation
Advance compositing and animationAdvance compositing and animation
Advance compositing and animation
 
論文輪読資料「Gated Feedback Recurrent Neural Networks」
論文輪読資料「Gated Feedback Recurrent Neural Networks」論文輪読資料「Gated Feedback Recurrent Neural Networks」
論文輪読資料「Gated Feedback Recurrent Neural Networks」
 
Fuzzieee-98-final
Fuzzieee-98-finalFuzzieee-98-final
Fuzzieee-98-final
 
Travelling salesman problem
Travelling salesman problemTravelling salesman problem
Travelling salesman problem
 
Practical Reinforcement Learning with TensorFlow
Practical Reinforcement Learning with TensorFlowPractical Reinforcement Learning with TensorFlow
Practical Reinforcement Learning with TensorFlow
 
Different techniques for speech recognition
Different  techniques for speech recognitionDifferent  techniques for speech recognition
Different techniques for speech recognition
 
P, NP, NP-Complete, and NP-Hard
P, NP, NP-Complete, and NP-HardP, NP, NP-Complete, and NP-Hard
P, NP, NP-Complete, and NP-Hard
 
End sem
End semEnd sem
End sem
 
Cb25464467
Cb25464467Cb25464467
Cb25464467
 

Similar to Dynamic Memory Networks for Dialogue Topic Tracking

Deep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT ConferenceDeep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT ConferenceBill Liu
 
Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...
Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...
Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...Universitat Politècnica de Catalunya
 
Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020
Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020
Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020Universitat Politècnica de Catalunya
 
PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...
PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...
PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...Jisang Yoon
 
PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018
PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018
PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018Universitat Politècnica de Catalunya
 
[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)Susang Kim
 
Hate speech detection
Hate speech detectionHate speech detection
Hate speech detectionNASIM ALAM
 
Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...
Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...
Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...Pierre Laffitte
 
FEC & File Multicast
FEC & File MulticastFEC & File Multicast
FEC & File MulticastYoss Cohen
 
Towards efficient content dissemination over DTN
Towards efficient content dissemination over DTNTowards efficient content dissemination over DTN
Towards efficient content dissemination over DTNAmir Krifa
 
On Optimization of Network-coded Scalable Multimedia Service Multicasting
On Optimization of Network-coded Scalable Multimedia Service MulticastingOn Optimization of Network-coded Scalable Multimedia Service Multicasting
On Optimization of Network-coded Scalable Multimedia Service MulticastingAndrea Tassi
 
Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...
Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...
Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...Universitat Politècnica de Catalunya
 
PPT Image Analysis(IRDE, DRDO)
PPT Image Analysis(IRDE, DRDO)PPT Image Analysis(IRDE, DRDO)
PPT Image Analysis(IRDE, DRDO)Nidhi Gopal
 
Titan X Research Paper
Titan X Research PaperTitan X Research Paper
Titan X Research PaperJennifer Wood
 
DL for setence classification project presentation
DL for setence classification project presentationDL for setence classification project presentation
DL for setence classification project presentationHoàng Triều Trịnh
 
Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Julien SIMON
 
Applying Deep Learning Machine Translation to Language Services
Applying Deep Learning Machine Translation to Language ServicesApplying Deep Learning Machine Translation to Language Services
Applying Deep Learning Machine Translation to Language ServicesYannis Flet-Berliac
 

Similar to Dynamic Memory Networks for Dialogue Topic Tracking (20)

Deep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT ConferenceDeep Learning for Speech Recognition in Cortana at AI NEXT Conference
Deep Learning for Speech Recognition in Cortana at AI NEXT Conference
 
Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...
Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...
Recurrent Neural Networks I (D2L2 Deep Learning for Speech and Language UPC 2...
 
Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020
Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020
Recurrent Neural Networks RNN - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...
PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...
PPT - Enhancing the Locality and Breaking the Memory Bottleneck of Transforme...
 
PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018
PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018
PixelCNN, Wavenet, Normalizing Flows - Santiago Pascual - UPC Barcelona 2018
 
[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)[Paper] Multiscale Vision Transformers(MVit)
[Paper] Multiscale Vision Transformers(MVit)
 
Hate speech detection
Hate speech detectionHate speech detection
Hate speech detection
 
Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...
Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...
Deep Neural Networks for Automatic detection of Sreams and Shouted Speech in ...
 
NP-Completeness - II
NP-Completeness - IINP-Completeness - II
NP-Completeness - II
 
B.tech_project_ppt.pptx
B.tech_project_ppt.pptxB.tech_project_ppt.pptx
B.tech_project_ppt.pptx
 
FEC & File Multicast
FEC & File MulticastFEC & File Multicast
FEC & File Multicast
 
Towards efficient content dissemination over DTN
Towards efficient content dissemination over DTNTowards efficient content dissemination over DTN
Towards efficient content dissemination over DTN
 
On Optimization of Network-coded Scalable Multimedia Service Multicasting
On Optimization of Network-coded Scalable Multimedia Service MulticastingOn Optimization of Network-coded Scalable Multimedia Service Multicasting
On Optimization of Network-coded Scalable Multimedia Service Multicasting
 
Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...
Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...
Recurrent Neural Networks II (D2L3 Deep Learning for Speech and Language UPC ...
 
PPT Image Analysis(IRDE, DRDO)
PPT Image Analysis(IRDE, DRDO)PPT Image Analysis(IRDE, DRDO)
PPT Image Analysis(IRDE, DRDO)
 
Titan X Research Paper
Titan X Research PaperTitan X Research Paper
Titan X Research Paper
 
DL for setence classification project presentation
DL for setence classification project presentationDL for setence classification project presentation
DL for setence classification project presentation
 
Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)Deep Dive on Deep Learning (June 2018)
Deep Dive on Deep Learning (June 2018)
 
Applying Deep Learning Machine Translation to Language Services
Applying Deep Learning Machine Translation to Language ServicesApplying Deep Learning Machine Translation to Language Services
Applying Deep Learning Machine Translation to Language Services
 
CNN for modeling sentence
CNN for modeling sentenceCNN for modeling sentence
CNN for modeling sentence
 

More from Seokhwan Kim

The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)Seokhwan Kim
 
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Seokhwan Kim
 
The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)Seokhwan Kim
 
Natural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionNatural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionSeokhwan Kim
 
The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)Seokhwan Kim
 
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Seokhwan Kim
 
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Seokhwan Kim
 
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...Seokhwan Kim
 
Sequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog StatesSequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog StatesSeokhwan Kim
 
Wikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingWikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingSeokhwan Kim
 
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...Seokhwan Kim
 
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...Seokhwan Kim
 
MMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionMMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionSeokhwan Kim
 
A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...Seokhwan Kim
 
A spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessA spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessSeokhwan Kim
 
An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...Seokhwan Kim
 
An Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionAn Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionSeokhwan Kim
 
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템Seokhwan Kim
 
A Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionA Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionSeokhwan Kim
 
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...Seokhwan Kim
 

More from Seokhwan Kim (20)

The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)The Eighth Dialog System Technology Challenge (DSTC8)
The Eighth Dialog System Technology Challenge (DSTC8)
 
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punc...
 
The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)The Fifth Dialog State Tracking Challenge (DSTC5)
The Fifth Dialog State Tracking Challenge (DSTC5)
 
Natural Language in Human-Robot Interaction
Natural Language in Human-Robot InteractionNatural Language in Human-Robot Interaction
Natural Language in Human-Robot Interaction
 
The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)The Fourth Dialog State Tracking Challenge (DSTC4)
The Fourth Dialog State Tracking Challenge (DSTC4)
 
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constra...
 
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
Towards Improving Dialogue Topic Tracking Performances with Wikification of C...
 
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
A Composite Kernel Approach for Dialog Topic Tracking with Structured Domain ...
 
Sequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog StatesSequential Labeling for Tracking Dynamic Dialog States
Sequential Labeling for Tracking Dynamic Dialog States
 
Wikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic TrackingWikipedia-based Kernels for Dialogue Topic Tracking
Wikipedia-based Kernels for Dialogue Topic Tracking
 
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
A Graph-based Cross-lingual Projection Approach for Spoken Language Understan...
 
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
A Graph-based Cross-lingual Projection Approach for Weakly Supervised Relatio...
 
MMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognitionMMR-based active machine learning for Bio named entity recognition
MMR-based active machine learning for Bio named entity recognition
 
A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...A semi-supervised method for efficient construction of statistical spoken lan...
A semi-supervised method for efficient construction of statistical spoken lan...
 
A spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information accessA spoken dialog system for electronic program guide information access
A spoken dialog system for electronic program guide information access
 
An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...An alignment-based approach to semi-supervised relation extraction including ...
An alignment-based approach to semi-supervised relation extraction including ...
 
An Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information ExtractionAn Alignment-based Pattern Representation Model for Information Extraction
An Alignment-based Pattern Representation Model for Information Extraction
 
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
EPG 정보 검색을 위한 예제 기반 자연어 대화 시스템
 
A Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation DetectionA Cross-Lingual Annotation Projection Approach for Relation Detection
A Cross-Lingual Annotation Projection Approach for Relation Detection
 
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
A Cross-lingual Annotation Projection-based Self-supervision Approach for Ope...
 

Recently uploaded

Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsAhmed Mohamed
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprisepreethippts
 
Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf
Innovate and Collaborate- Harnessing the Power of Open Source Software.pdfInnovate and Collaborate- Harnessing the Power of Open Source Software.pdf
Innovate and Collaborate- Harnessing the Power of Open Source Software.pdfYashikaSharma391629
 
Cloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEECloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEEVICTOR MAESTRE RAMIREZ
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odishasmiwainfosol
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commercemanigoyal112
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identityteam-WIBU
 
Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Mater
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions
 
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxRTS corp
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Rob Geurden
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
SpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at RuntimeSpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at Runtimeandrehoraa
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based projectAnoyGreter
 

Recently uploaded (20)

Unveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML DiagramsUnveiling Design Patterns: A Visual Guide with UML Diagrams
Unveiling Design Patterns: A Visual Guide with UML Diagrams
 
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprise
 
Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf
Innovate and Collaborate- Harnessing the Power of Open Source Software.pdfInnovate and Collaborate- Harnessing the Power of Open Source Software.pdf
Innovate and Collaborate- Harnessing the Power of Open Source Software.pdf
 
Cloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEECloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEE
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
Cyber security and its impact on E commerce
Cyber security and its impact on E commerceCyber security and its impact on E commerce
Cyber security and its impact on E commerce
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identity
 
Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)Ahmed Motair CV April 2024 (Senior SW Developer)
Ahmed Motair CV April 2024 (Senior SW Developer)
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
 
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
 
Advantages of Odoo ERP 17 for Your Business
Advantages of Odoo ERP 17 for Your BusinessAdvantages of Odoo ERP 17 for Your Business
Advantages of Odoo ERP 17 for Your Business
 
A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
SpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at RuntimeSpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at Runtime
 
MYjobs Presentation Django-based project
MYjobs Presentation Django-based projectMYjobs Presentation Django-based project
MYjobs Presentation Django-based project
 

Dynamic Memory Networks for Dialogue Topic Tracking

  • 1. Dynamic Memory Networks for Dialogue Topic Tracking Seokhwan Kim Adobe Research, San Jose, CA, USA Dialogue Topic Tracking Categorizing the topic state at each time step f(t) =    B-{c ∈ C} if ut is at the beginning of a segment belongs to c, I-{c ∈ C} else if ut is inside a segment belongs to c, O otherwise, Examples of dialogue topic tracking Speaker Utterance (ut) f(t) Guide How can I help you? B-OPEN Tourist Can you recommend some good places to visit in Singa- pore? B-ATTR Guide Well if you like to visit an icon of Singapore, Merlion will be a nice place to visit. I-ATTR Tourist Okay. But I’m particularly interested in amusement parks. B-ATTR Guide Then, what about Universal Studio? I-ATTR Tourist Good! How can I get there from Orchard Road by public transportation? B-TRSP Guide You can take the red line train from Orchard and transfer to the purple line at Dhoby Ghaut. Then, you could reach HarbourFront where Sentosa Express departs. I-TRSP Tourist How long does it take in total? I-TRSP Guide It’ll take around half an hour. I-TRSP Tourist Alright. I-TRSP Guide You could spend a whole afternoon at the park by its closing time at 6pm. B-ATTR Tourist Sounds good! I-ATTR Guide Then, I recommend you enjoy dinner at the riverside on the way back. B-FOOD Tourist What do you recommend there? I-FOOD Guide If you like spicy foods, you must try chili crab which is one of our favorite dishes. I-FOOD Tourist Great! I’ll try that. I-FOOD Baselines: CNN and RCNN (Kim et al., 2016) CNN RCNN Inputs ut-1 ut ut-w+1 … ut-2ut-2 Convolutional layer Max pooling layer Prediction yt Inputs ut-w+1 … ut-1 ut ut-2ut-2 Convolutional layer Max pooling layer Max pooling layer Recurrent layer ht-w+1 … ht-2 ht-1 ht Prediction yt Prediction yt Convolutional Neural Network (CNN) for Dialogue Topic Tracking Representing an utterance as a matrix with n rows of k-dimensional word vectors A convolutional filter has the same width k and a window size m as its height The maximum value is selected from each feature map in max pooling layer The values from max pooling are forwarded to the fully-connected softmax layer Recurrent CNN (RCNN) for Dialogue Topic Tracking Each feature vector generated after the max pooling layers in the CNN architecture is connected to the recurrent layers in the RNN architecture Proposed Model: Dynamic Memory Network Dynamic Memory Network (DMN) for Dialogue Topic Tracking Inputs ut-w+1 … ut-1 ut ut-2ut-2 Convolutional layer Dynamic Memories Prediction yt Prediction yt h1 t-w+1 h2 t-w+1 hm t-w+1… h1 t-2 h2 t-2 hm t-2… h1 t-1 h2 t-1 hm t-1…h1 t-1 h2 t-1 hm t-1… h1 t h2 t hm t…h1 t h2 t hm t… … … … Max pooling Proposed Model: Dynamic Memory Network Our models represent the latent dialogue state at each given time step as a set of read-writable memory slots Each memory slot is updated through a given dialogue sequence by the content-based operations in gated recurrent networks Gating mechanisms Single Gate (Henaff et al. 2016) Update and Reset Gates Cross-slot Interactions zj i σ uT i wj + uT i hj i−1 σ k αkj z uT i wk + βkj z uT i hk i−1 rj i - σ uT i Wr wj + uT i Ur hj i−1 σ k αkj r uT i wk + βkj r uT i hk i−1 ˜hj i tanh Uhj i−1 + Vwj + Wui tanh U rj i ◦ hj i−1 + Vwj + Wui hj i 1 − zj i ◦ hj i−1 + zj i ◦ ˜hj i Evaluation: Data TourSG corpus Human-human mixed initiative dialogues 35 sessions, 21 hours, 31,034 utterances Manually annotated with eight topic categories ‘attraction’, ‘transportation’, ‘food’, ‘accommodation’, ‘shopping’, ‘opening’, ‘closing’, ‘other’ 15 classes: ({B-, I-} × {c : c ∈ C; and c = ‘other’}) ∪ {O} Data Statistics Set # sessions # segments # utterances Train 14 2,104 12,759 Dev 6 700 4,812 Test 15 2,210 13,463 Total 35 5,014 31,034 Evaluation: Implementation Details Word Embedding Initialized with the pre-trained word2vec on 2.9M sentences from travel forum Fine-tuned while the whole model is trained Convolutional Layer Learned 100 feature maps for each of three different filter sizes {3, 4, 5} For CNN, applied over the current, previous, and history utterances w = 10 For RCNN and DMN, applied for each single utterance Recurrent Layer RCNNs We compared two variants: Vanilla RNNs and Gated Recurrent Units (GRUs) The hidden layer dimensions were 150 for the vanilla RNN and 50 for the GRU DMNs Three dynamic memory networks were trained based on the proposed gating mechanisms The number of memory slots were m = 5 for the first two distributed models and m = 10 for the other with cross-slot interactions Model Training Adam optimizer by minimizing the categorical cross entropy loss on softmax With mini-batch size of 50 and dropout after max pooling with the rate of 0.25 Stopped training after 150 epochs where the CNN baseline has been saturated. Evaluation: Results Sequential Labelling Segmentation Models P R F Pk WD CNN 0.6691 0.6861 0.6775 0.3799 0.4884 RCNN (RNN) 0.6825 0.6572 0.6696 0.3970 0.4634 RCNN (GRU) 0.6936 0.6826 0.6880 0.3888 0.4619 DMN (single) 0.6877 0.7105 0.6989 0.3782 0.4393 DMN (reset & update) 0.6959 0.7035 0.6997 0.3781 0.4427 DMN (cross-slot) 0.7008 0.7090 0.7049† 0.3532‡ 0.4223‡ CNN RCNN Dynamic Memory 0 500 1000 1500 2000 2500 3000 3500 4000 4500 Numberoferrors missing extraneous wrong boundary wrong category Slot 0 Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Slot 8 Slot 9 B-ATTR B-TRSP B-FOOD B-ACCO B-SHOP I-ATTR I-TRSP I-FOOD I-ACCO I-SHOP 0.6 0.7 0.8 0.9 345 Park Avenue, San Jose, CA 95110, USA Email: seokim@adobe.com