This document summarizes a presentation on unpaired image-to-image translation using cycle-consistent adversarial networks. It discusses the motivation for this approach, which is to perform image translation without needing paired training data, using a cycle consistency loss to ensure the translated image is similar to the original. The method uses GANs with this additional cycle consistency loss. Experiments demonstrated it can perform the translation but sometimes fails to capture semantic features fully.

1. D S A I Lab.
2021.05.10.
Unpaired Image-to-Image Translation using
Cycle-Consistent Adversarial Networks
Zhu et al., (Berkeley univ.), ICCV 2017
숭실대학교 소프트웨어학과 DSAI Lab. 지승현 발표
1

2. D S A I Lab.
Index
1. Motivation
2. Method
3. Experiments
4. Summary
2

3. D S A I Lab.
1. Motivation
– Image translation task
– A task that translate the given image into another one in specific style.
– If there’s enough paired data, this can be done with supervised learning.
– However, getting those is costly.
3

4. D S A I Lab.
1. Motivation
– “Image generation” approach can be a solution.
– We can assume there’s a pattern in image set 𝒀.
– There’s a underlying relationship between the domains.
– If we can learn
𝐆: 𝐗 → 𝒀, 𝑷 𝒀 𝑿
, we can translate image x into y.
– However, generating images with GAN cannot guarantee
that translated image is similar with original image.
 So this paper proposed ‘Cycle Consistent Translation’
4

5. D S A I Lab.
1. Motivation
– Proposed method “Cycle consistency” is similar with “Autoencoder”.
– In image translation task, If there’s a translation function
𝑮: 𝑿 → 𝒀 𝑭: 𝒀 → 𝑿
– then cycle consistency loss encourages
𝑭 𝑮 𝒙 ≈ 𝒙 𝑭 𝑮 𝒚 ≈ 𝒚
– In Ideal Autoencoder,
𝑬𝒏𝒄𝒐𝒅𝒆𝒓 𝒙 = 𝒉𝒙 𝑫𝒆𝒄𝒐𝒅𝒆𝒓 𝒉𝒙 = 𝒙
– The key of training process this architecture* is a loss function.
5
* Denoising autoencoder, … etc

6. D S A I Lab.
2. Method
– Zhu et al. proposed GAN with Cycle consistency loss
– GAN is good idea for training generator.
– However, GAN loss is for generating look-alike-y image, not for assuring y is from x.
– Which means that this can generate random image which meets given condition.
 So Zhu et al. argued that Cycle consistency loss can relieve this problem.
6

7. D S A I Lab.
2. Method
– Proposed ‘Cycle consistency loss’ :
– This loss helps F(x) to look like that is originated from x.
– Full objective is :
7

8. D S A I Lab.
3. Experiments
8

9. D S A I Lab.
3. Experiments
– Some failure cases
– Maybe new method that captures semantic features well is needed.
9

10. D S A I Lab.
4. Summary
– In image translation, GAN loss cannot assure translated image is
originated from original image. (can generate random image)
– So proposing method ‘cycle consistency loss’ can help model keep
input information.
10