SlideShare a Scribd company logo

Understanding Back-Translation at Scale

•
•
S
Satoru Katsumata

- 首都大 小町研 EMNLP 2018 読み会 - Understanding Back-Translation at Scale. Edunov et al., EMNLP2018 を紹介 - paper URL: https://www.aclweb.org/anthology/D18-1045.pdf

Science
1 of 29
Download to read offline

 


 
 
 !2

 
 
 !3 
 
 
 
 


 
 
 !4 
 
 
 
 


 
 
 
 
 
 !5 
 
 
 
 


 
 
 
 
 
 !6 
 
 
 
 
 
 


 
 
 
 
 
 !7 
 
 
 
 


 
 
 
 
 
 !8 
 
 
 
 
 


 
 
 
 
 
 !9 
 
 
 
 
 
 
 
 


 
 
 
 
 
 
 
 !10

 
 
 
 
 
 
 !11 yt−1 yt−1 … … yt−2 … … yt

 
 
 
 
 
 
 
 !12 
 
 🤔 


 
 
 
 
 
 
 !13 yt−1 yt−1 … … yt−2 … … yt yt−1 yt−1 … … yt−2 … … yt 
 
 
 


 
 
 
 
 
 
 !14 


 
 
 
 
 
 
 !15 


 
 
 
 
 
 
 !16 🤔 
 


 
 
 
 
 
 
 
 !17

 !18

 !19 
 
 
 


 
 
 
 !20 
 
 🤔 
 🤔

 
 
 
 
 
 
 
 
 !21

 
 
 
 
 
 
 
 
 !22 
 
 
 
 


 
 !23 
 
 
 
 


 
 
 
 
 
 !24

 
 
 
 
 
 !25 
 
 
 
 


 
 
 
 
 
 !26

 !27

 🤔 
 !28

 !29

Recommended

тодосієнко а.
тодосієнко а.тодосієнко а.
тодосієнко а.
Olena Pyzaenko
 
Exploiting Monolingual Data at Scale for Neural Machine Translation
Exploiting Monolingual Data at Scale for Neural Machine TranslationExploiting Monolingual Data at Scale for Neural Machine Translation
Exploiting Monolingual Data at Scale for Neural Machine Translation
Satoru Katsumata
 
Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...
Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...
Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...
Satoru Katsumata
 
How Contextual are Contextualized Word Representations?
How Contextual are Contextualized Word Representations?How Contextual are Contextualized Word Representations?
How Contextual are Contextualized Word Representations?
Satoru Katsumata
 
Word-node2vec
Word-node2vecWord-node2vec
Word-node2vec
Satoru Katsumata
 
Corpora Generation for Grammatical Error Correction
Corpora Generation for Grammatical Error CorrectionCorpora Generation for Grammatical Error Correction
Corpora Generation for Grammatical Error Correction
Satoru Katsumata
 
2018年度レトリバインターン参加報告
2018年度レトリバインターン参加報告2018年度レトリバインターン参加報告
2018年度レトリバインターン参加報告
Satoru Katsumata
 
On the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionOn the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary Induction
Satoru Katsumata
 

Recommended

тодосієнко а.
тодосієнко а.тодосієнко а.
тодосієнко а.
Olena Pyzaenko
 
Exploiting Monolingual Data at Scale for Neural Machine Translation
Exploiting Monolingual Data at Scale for Neural Machine TranslationExploiting Monolingual Data at Scale for Neural Machine Translation
Exploiting Monolingual Data at Scale for Neural Machine Translation
Satoru Katsumata
 
Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...
Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...
Incorporating Syntactic and Semantic Information in Word Embeddings using Gra...
Satoru Katsumata
 
How Contextual are Contextualized Word Representations?
How Contextual are Contextualized Word Representations?How Contextual are Contextualized Word Representations?
How Contextual are Contextualized Word Representations?
Satoru Katsumata
 
Word-node2vec
Word-node2vecWord-node2vec
Word-node2vec
Satoru Katsumata
 
Corpora Generation for Grammatical Error Correction
Corpora Generation for Grammatical Error CorrectionCorpora Generation for Grammatical Error Correction
Corpora Generation for Grammatical Error Correction
Satoru Katsumata
 
2018年度レトリバインターン参加報告
2018年度レトリバインターン参加報告2018年度レトリバインターン参加報告
2018年度レトリバインターン参加報告
Satoru Katsumata
 
On the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary InductionOn the Limitations of Unsupervised Bilingual Dictionary Induction
On the Limitations of Unsupervised Bilingual Dictionary Induction
Satoru Katsumata
 
Guiding neural machine translation with retrieved translation pieces
Guiding neural machine translation with retrieved translation piecesGuiding neural machine translation with retrieved translation pieces
Guiding neural machine translation with retrieved translation pieces
Satoru Katsumata
 
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...
Satoru Katsumata
 
Memory-augmented Neural Machine Translation
Memory-augmented Neural Machine TranslationMemory-augmented Neural Machine Translation
Memory-augmented Neural Machine Translation
Satoru Katsumata
 
Lexically constrained decoding for sequence generation using grid beam search
Lexically constrained decoding for sequence generation using grid beam searchLexically constrained decoding for sequence generation using grid beam search
Lexically constrained decoding for sequence generation using grid beam search
Satoru Katsumata
 
Deep Neural Machine Translation with Linear Associative Unit
Deep Neural Machine Translation with Linear Associative UnitDeep Neural Machine Translation with Linear Associative Unit
Deep Neural Machine Translation with Linear Associative Unit
Satoru Katsumata
 
A convolutional encoder model for neural machine translation
A convolutional encoder model for neural machine translationA convolutional encoder model for neural machine translation
A convolutional encoder model for neural machine translation
Satoru Katsumata
 

More Related Content

More from Satoru Katsumata

More from Satoru Katsumata (6)

Guiding neural machine translation with retrieved translation pieces
Guiding neural machine translation with retrieved translation piecesGuiding neural machine translation with retrieved translation pieces
Guiding neural machine translation with retrieved translation pieces
 
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Mo...
 
Memory-augmented Neural Machine Translation
Memory-augmented Neural Machine TranslationMemory-augmented Neural Machine Translation
Memory-augmented Neural Machine Translation
 
Lexically constrained decoding for sequence generation using grid beam search
Lexically constrained decoding for sequence generation using grid beam searchLexically constrained decoding for sequence generation using grid beam search
Lexically constrained decoding for sequence generation using grid beam search
 
Deep Neural Machine Translation with Linear Associative Unit
Deep Neural Machine Translation with Linear Associative UnitDeep Neural Machine Translation with Linear Associative Unit
Deep Neural Machine Translation with Linear Associative Unit
 
A convolutional encoder model for neural machine translation
A convolutional encoder model for neural machine translationA convolutional encoder model for neural machine translation
A convolutional encoder model for neural machine translation
 

Understanding Back-Translation at Scale