首都大 小町研 研究室内論文紹介（M1 後期）

1. PARANMT-50M: Pushing the Limits of Paraphrastic Sentence
Embeddings with Millions of Machine Translations
2018/10/24 M1 Hiroki Shimanaka
John Wie)ng and Kevin Gimpel
ACL 2018

2. lNMT
l
l STS (Semantic Textual Similarity )
l SemEval STS
1

3. l
l PPDB
l NMT
l
NLP
2

4. PARANMT-50M
l NMT
l NMT
l ( CzEng
) NMT
3

5. l
10
l
l Europarl
lCzEng
4

6. 5

7. PARANMT-50M
6
l2 5
100

8. lPARANMT-50M
l 4
ØWORD
ØTRIGRAM
ØLSTM
ØBLSTM
7

9. 8
!l ⟨s,sʹ⟩
$l
%l
l&
l!' ⊆ !

10. lmini-batch
mini-batch
l mega-batch
! mini-
batch mega-batch
!
9

11. lPARANMT-50M Sem-Eval STS
2012-2016 STS Benchmark (2017)
lSTS 2 0-5
0
5
10

12. l PARAGRAM-
SL999 embeddings [Wieting et al., 2015]
lmini-batch 100
l! 0.4
lEpoch 5
lOptimizer Adam lr: 0.001
11

13. 12
CzEng

14. l50M 3 30
Øthe length-normalized translation score
Øtrigram overlap [Wieting et al., 2017]
Øthe paraphrase score [Wieting et al., 2016]
13

15. mega-batch
14

16. STS
15
lBLEU METEOR
lSimpWiki PARANMT-50M

17. STS Benchmark
16
WORD, TRIGRAMl
(concatna4on)

18. 17
l
l

19. SCPN
18
lSCPN (Syntactically Controlled Paraphrase Net-work) [Iyyer
et al., 2018]

20. 19
l
l

21. l50M
l
l
l
20