This document discusses using adversarial training methods to improve neural machine translation. Specifically, it explores training a Transformer model with both original inputs and their "doubly adversarial" corrupted versions to learn robust representations. Evaluation on two translation tasks showed this approach improved BLEU scores over the baseline Transformer model, demonstrating the effectiveness of adversarial training for more robust neural machine translation.

1. Robust Neural Machine
Translation with Doubly
Adversarial Inputs
Yong Cheng, Lu Jiang, Wolfgang Macherey
ACL 2019
2019/9/28
1

2. • NMT
2
- he she

3. •
•
–
3
Der … geladenen … Der … vorgeladenen …<BOS> … summoned …
The … continue …
→
<BOS> … invited …
The … continue …

4. • xi
•
xi’
4
xi gxi
→ xi

5. • xi
•
xi’
5
xi gxi
→ xi

6. •
•
–
•
–
• P(j)
• M δ
6

7. •
•
– 10
•
–
10
7

8. •
•
–
NMT
– NMT
– NMT
8

9. •
– - 120 LDC
– - 450 WMT14
• Transformer [Vaswani+ 17]
• BPE
– - 6
– - 3 2
•
9

10. 10
-
-
Vaswani+ 17

11. •
• BackTranskation [Sennrich+ 16]
11

12. •
12
BLEU
BLEU
BLEU

13. Ablation test
•
13
→ Teacher forcing

14. • NMT
–
– Transformer [Vaswani+ 17]
–
•
– Teacher
forcing
14