1. Two papers on unsupervised domain adaptation were presented at ICML2018: "Learning Semantic Representations for Unsupervised Domain Adaptation" and "CyCADA: Cycle-Consistent Adversarial Domain Adaptation". 2. The CyCADA paper uses cycle-consistent adversarial domain adaptation with cycle GAN to translate images at the pixel level while also aligning representations at the semantic level. 3. The semantic representation paper uses semantic alignment and introduces techniques like adding noise to improve over previous semantic alignment methods.

1. 1
DEEP LEARNING JP
[DL Papers]
http://deeplearning.jp/
"CyCADA: Cycle-Consistent Adversarial Domain Adaptation"
& "Learning Semantic Representations for Unsupervised
Domain Adaptation" (ICML2018 )
Presentater: Kei Akuzawa, Matsuo Lab. M2

2. •
•
•
• ICML2018
• [Hoffman+] CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• [Xie+] Learning Semantic Representations for Unsupervised Domain Adaptation

3. • : Source Target
Source
Target
• :
• Source: (X_s, Y_s)
• Target:
• Unsupervised Domain Adaptation: (X_t) <-
• Supervised Domain Adaptation: (X_t, Y_t)

4. Source Target
Ganin+ 2016

5. • :
•
•
•
• Bengio ……(Talk at the ICML'2018 Workshop on Learning with Limited Labels, July 13th, 2018.)
• Current ML theory is strongly dependent on the iid assumption
• Real-life applications often require generalizations in regimes not seen during training
• Humans can project themselves in situations they have never been (e.g. imagine
being on another planet, or going through exceptional events like in many movies)
,

6. •
• Ganin+ 2016
• Tzeng+ 2017
• Saito+ 2018
•
• Taigman+ 2017
• Shrivastava+ 2017
• Hosseini-Asl+ 2018
Ganin+ 2016
Shrivastava+ 2017

7. :
• Source
q(y|x)
( !"[$% & ' ] ≠ !*[$* & ' ] )
• z( )
p(z|x) p(y|z)
• z ?:
• MMD,
x
z
dy
z

8. :
Ganin+ 2016
• :
• Discriminator (z)
• Encoder Discriminator (z)
• Min-Max (z)

9. !!
•
• Fair Prediction: (Domain)
• Style Transfer:
• Domain Generalization:

10. :
• Target y
•
•
• Unsupervised Image Translation
(X_s) (X_t^')
(X_t^', y_s)

11. • Learning Semantic Representations for Unsupervised Domain Adaptation
• + ( )
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• + ( )

12. 1.
• Title:
Learning Semantic Representations for Unsupervised Domain Adaptation
• Authors:
Shaoan Xie, Zibin Zheng, Liang Chen, Chuan Chen
• Info:
ICML2018 accepted (oral)
• Abstract:
[ , ]
Noisy
Centroid Noise
[ , ]

13. [ , ]
Saito+ 2018

14. :
•
•
H-divergence
( )

15. : C
!!
( C)
Semantic Alignment

16. • Semantic Alignment [saito+ 2017]
1.
2.
3. (w/ )
4.
• (Noisy)
•
•
• Centroid

17. Centroid
( )
Centroid

18. :
• DAN Semantic Loss
• Centroid
•

19. • MNIST-USPS-SVHN
• DAN
•

20. • [ ]
(Semantic Alignment )
• Semantic Alignment Noisy
• Noise

21. 2.
• Title:
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Authors:
Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei
Efros, Trevor Darrell
• Info:
ICML2018 accepted (oral)
• Abstract:
(Pixel)
Domain Translation CycleGAN

22. •
• Semantic Alignment
•
• c.f. ( ) [Jo and
Bengio, 2017]
• ( )
•
•
• Cycle-GAN
•
• Cycle-GAN
•

23. 4 Component
• Cycle-GAN
• Semantic Consistency
• Classifier
• Feature Adaptation

24. : Cycle-GAN
Cycle-GAN
->

25. : Semantic Consistency Loss
(f_s pre-train )

26. : Classifier

27. : Feature Adaptation
Domain Adversarial Network

28. :
• SVHN -> MNIST
Pixel
Adaptation Feature
Adaptation
• Pixel Feature

29. : Cycle Loss Semantic Loss
• SVHN -> MNIST
Semantic Alignment
• Semantic Segmentation( )
Cycle Loss
Cycle

30. : Semantic Segmentation

31. •
• Cycle-GAN Cycle Loss Semantic
Loss

32. •
•
•
• [Saito+ 2018b]:
• [Hosseini-Asl+ 2018]:

33. References
• Y.Ganin,E.Ustinova,H.Ajakan,P.Germain,H.Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky.
Domain- adversarial training of neural networks. JMLR, 17(59):1–35, 2016.
• Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T. Adversarial discriminative domain adaptation. In
Computer Vision and Pattern Recognition (CVPR), 2017.
• Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada. Asymmetric Tri-training for Unsupervised Domain
Adaptation. The 34th International Conference on Machine Learning (ICML 2017), 2017.
• Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, Tatsuya Harada. Maximum Classifier Discrepancy for
Unsupervised Domain Adaptation. The 31th IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR 2018), 2018, (oral).
• Kuniaki Saito, Shohei Yamamoto, Yoshitaka Ushiku, Tatsuya Harada, Open Set Domain Adaptation by
Backpropagation, arXiv, 2018b
• Taigman, Y., Polyak, A., and Wolf, L. Unsupervised cross-domain image generation. In International
Conference on Learning Representations, 2017a.
• Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., and Webb, R. Learning from simulated and
unsupervised im- ages through adversarial training. In The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), July 2017.
• Ehsan Hosseini-Asl, Yingbo Zhou, Caiming Xiong, Richard Socher, Augmented Cyclic Adversarial Learning for
Domain Adaptation, arxiv, 2018
• Jason Jo, Yoshua Bengio, Measuring the tendency of CNNs to Learn Surface Statistical Regularities, arxiv
2017, https://arxiv.org/abs/1711.11561