This document summarizes several papers on dual learning techniques for machine learning and natural language processing tasks. It discusses dual unsupervised learning (NIPS 2016), dual supervised learning (ICML 2017), dual inference for machine translation (IJCAI 2017), dual transfer learning (AAAI 2018), and model-level dual learning (ICML 2018). For each paper, it provides a brief overview of the task, methodology, and contributions. The overall document serves as a reference for recent work applying dual learning frameworks across different domains.

1. DEEP LEARNING JP
[DL Papers]
Dual Learning
Toru Fujino, SCSLAB, UTokyo
http://deeplearning.jp/
1

2. n Dual Unsupervised Learning (NIPS 2016)
n Dual Supervised Learning (ICML 2017)
n Dual Inference for Machine Translation (IJCAI 2017)
n Dual Transfer Learning (AAAI 2018)
n Model-Level Dual Learning (ICML 2018)
n
n ,
n
n + +

3. n Duality
n (f) (g)
n (f) (g)
n (f) (g)
n 2 !, # ,
n $: & → (, )((|&; -./)
n 1: ( → &, )(&|(; -/.)

4. n
n
n
n
n etc.
n Duality
n
n
n

5. n Dual Unsupervised Learning (NIPS 2016)
n Dual Supervised Learning (ICML 2017)
n Dual Inference for Machine for Machine Learning (IJCAI 2017)
n Dual Transfer Learning (AAAI 2018)
n Model-Level Dual Learning (ICML 2018)

6. Dual Unsupervised Learning
n
n 1M ~
n
n
n

7. Dual Unsupervised Learning
n ,
n , 1 1
Unsupervised Learning
n
n ) -> dog ->
n
I am a student.
!(#|%; '())
!(%|#; ')()

8. Dual Unsupervised Learning
n
n
!" = $%&( )
n
!) = log -(I am a student| ; 9)
n 2
! = :!" + 1 − : !)
I am a student.
-(>|?; 9@A)
-(?|>; 9A@)
①
②

9. Dual Unsupervised Learning
K
REINFORCE

10. Dual Unsupervised Learning
n
n , -> Dog ->
n
n (10%, 100%)
n

11. Dual Unsupervised Learning
n
n Large 100%
n Small 10%

12. Dual Unsupervised Learning
n

13. Dual Unsupervised Learning
n ,
Unsupervised Learning
n
n Future Work
n

14. n Dual Unsupervised Learning (NIPS 2016)
n Dual Supervised Learning (ICML 2017)
n Dual Inference for Machine for Machine Learning (IJCAI 2017)
n Dual Transfer Learning (AAAI 2018)
n Model-Level Dual Learning (ICML 2018)

15. Dual Supervised Learning
n Primal task, Dual task
! " #; %&'
!(#|"; %'&)
n ,
! " #; %&' = ! # "; %'& = ! #, " for any #, "
n
n

16. Dual Supervised Learning
n
!"#$%&'( = log - . + log - 0 .; 23( − log - 0 − log - . 0; 2(3
5
n task
ℒ 23( = 7
8∑[!7 ; .<; 23( , 0< + >3(!"#$%&'((.<, 0<; 23(, 2(3)]
ℒ 2(3 = 7
8∑[!7 ; .<; 23( , 0< + >3(!"#$%&'((.<, 0<; 23(, 2(3)]

17. Dual Supervised Learning
n f g

18. Dual Supervised Learning
n
n
n

19. Dual Supervised Learning

20. Dual Supervised Learning

21. Dual Supervised Learning
n

22. n Dual Unsupervised Learning (NIPS 2016)
n Dual Supervised Learning (ICML 2017)
n Dual Inference for Machine for Machine Learning (IJCAI 2017)
n Dual Transfer Learning (AAAI 2018)
n Model-Level Dual Learning (ICML 2018)

23. Dual Inference for Machine Learning
n Primal task, Dual task
!: # → %
&: % → #
n '( ), + , ',(), +) ,
+⋆
= 12&max
67∈9
'( ), +:
)⋆ = 12& min
=7∈>
',():, +)
n ! + ) , & +
)
n

24. Dual Inference for Machine Learning
n ,
(Dual Inference)
!⋆ = $%& min
*+∈-
./0 1, !3 + 1 − . /7(1, !3)
&⋆
= $%& min
:+∈;
</7 13
, ! + 1 − < /0(13
, !)

25. Dual Inference for Machine Learning
n Dual Inference
1. ! K
2. K
"#$ %, '((*) + 1 − " #/(%, '( * )

26. Dual Inference for Machine Learning
n Dual Inference /

27. Dual Inference for Machine Learning
n (α, β)

28. Dual Inference for Machine Learning
n

29. Dual Inference for Machine Learning
n

30. n Dual Unsupervised Learning (NIPS 2016)
n Dual Supervised Learning (ICML 2017)
n Dual Inference for Machine for Machine Learning (IJCAI 2017)
n Dual Transfer Learning (AAAI 2018)
n Model-Level Dual Learning (ICML 2018)

31. Dual Transfer Learning
n
!(#|%; '())
n ,
! # = ,
(∈.
! # %; '() !(%)
n
n

32. Dual Transfer Learning (AAAI 2018)
n ,
# $ = ∑# $ '; )*+ #(') ≈ /
0
∑#($|' 2 ; )*+), ' 2 ∼ #(')
n # $ '; )*+ , ' ∼ #(') ( ) ,
#('|$)
# $ = ∑# $ '; )*+ #(') ≈ /
0∑
# $ ' 2 ; )*+ # ' 2
# ' 2 $
, ' 2 ∼ #('|$; )+*)
n #('|$; )+*)
n

33. Dual Transfer Learning
n
n ! "

34. Dual Transfer Learning
n

35. Dual Transfer Learning
n

36. Dual Transfer Learning
n λ

37. Dual Transfer Learning
n !(#|%; '())

38. Dual Transfer Learning
n
n
n

39. n Improving Neural Machine Translation Models with Monolingual
Data (ACL 2016)
n Neural Machine Translation with Reconstruction (AAAI 2017)
n Iterative Back-Translation for Neural Machine Translation (WNMT
2018)
n Joint Training for Neural Machine Translation Models with
Monolingual Data (AAAI 2018)
n Unsupervised Machine Translation Using Monolingual Corpora
Only (ICLR 2018)

40. n
n Dual Unsupervised Learning
n Dual Inference
n Dual Transfer Learning
n + Dual Supervised Learning
n Model-Level Dual Learning
n
n