Abstract Obake-GAN (Perturbative GAN), which replaces convolution layers of existing convolutional GANs (DCGAN, WGAN-GP , BIGGAN, etc.) with perturbation layers that adds a fixed noise mask, is proposed. Compared with the convolutional GANs, the number of parameters to be trained is smaller, the convergence of training is faster, the inception score of generated images is higher, and the overall training cost is reduced. Algorithmic generation of the noise masks is also proposed, with which the training, as well as the generation, can be boosted with hardware acceleration. Obake-GAN is evaluated using conventional datasets (CIFAR10, LSUN, ImageNet), both in the cases when a perturbation layer is adopted only for Generators and when it is introduced to both Generator and Discriminator . 修士論文「Obake-GAN: GAN with Perturbation Layers」の発表資料 GANの畳込層の代わりに摂動層を導入し、 ・Generator 学習パラメータ52%削減 ・Discriminator 学習パラメータ87%削減 ・ImageNetでInception Score 45%改善 ・学習の収束を高速化

1. Obake-GAN:
GAN with Perturbation Layers
obake2ai@gmail.com
@obake_ai

2. 2
ImageNet
WGAN-gp
[9]
Obake-GAN
(PGv1 vs PD)
iter 0 iter 1,000 iter 100,000

3. 3

4. 4

5. 5
VS

6. 6
VS

7. 7

8. 8

9. 9

10. 10

11. 11
1.Upsampling 2.Zero Padding 3.Convolution
Input
Convolution Filters Output
ReLU
4.Activation
Trainable Parameters
(Nearest Neighbor)

12. 12
z
G(z)
D(x),
D(G(z))
x

13. 13
z
G(z)
D(x),
D(G(z))
x

14. 14
z
G(z)
D(x),
D(G(z))
x
z
G(z) (Fake)
x (True)
D(x)
D(G(z))

15. 15
z
G(z)
D(x),
D(G(z))
x
z
G(z) (Fake)
x (True)
D(x)
D(G(z))

16. 16

17. 17

18. 18

19. 19

20. 20

21. 21

22. 22

23. 23

24. 24

25. 25

26. 26

27. 27
1.Upsampling 2.Zero Padding 3.Convolution
Input
Convolution Filters Output
ReLU
4.Activation
(Nearest Neighbor)
Trainable Parameters

28. 28
1.Upsampling 2.Zero Padding 3.Convolution
Input
Convolution Filters Output
ReLU
4.Activation
(Nearest Neighbor)
Trainable Parameters

29. 29

30. 30

31. 31

32. 32

33. 33

34. 34

35. 35

36. 36

37. 37

38. 38
z
G(z)
D(x),
D(G(z))
x

39. 39
1.Upsampling 2.Zero Padding 3.Convolution
Input
Convolution Filters Output
ReLU
4.Activation
(Nearest Neighbor)
Trainable Parameters

40. 40
MNIST

41. 41
1.Upsampling 2.Zero Padding 3.Convolution
Input
Convolution Filters Output
ReLU
4.Activation
(Nearest Neighbor)
Trainable Parameters

42. 42
Trainable
Parameters
Input
Fixed Perturbation Masks
(non-trainable)
Linear Weights
Output
ReLU
1.Perturbation 2.Activation 3.Linear Combination

43. 43
MNIST

44. 44
MNIST cifar10

45. 45
Trainable
Parameters
Input
Fixed Perturbation Masks
(non-trainable)
Linear Weights
Output
ReLU
1.Perturbation 2.Activation 3.Linear Combination

46. 46

47. 47
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

48. 48
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

49. 49
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

50. 50
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

51. 51
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

52. 52
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

53. 53
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input
Convolution Filters
Trainable Parameters
Fixed Perturbation Masks
(non-trainable)

54. 54
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input
Convolution Filters
Trainable Parameters
Trainable
Parameters
Linear Weights
p
q
k
Ratio =

55. 55

56. 56

57. 57
G: Generator x: Fake Image
p(y|x): x y
p(y): y
[p||q]: p,q KL Divergence

58. 58
G: Generator x: Fake Image
p(y|x): x y
p(y): y
[p||q]: p,q KL Divergence

59. 59
G: Generator x: Fake Image
p(y|x): x y
p(y): y
[p||q]: p,q KL Divergence

60. 60
G: Generator x: Fake Image
p(y|x): x y
p(y): y
[p||q]: p,q KL Divergence

61. 61
Perturbation
ReLU
BatchNorm
Linear Comb
Perturbation
ReLU
BatchNorm
Linear Comb
Bilinear
Perturbation
ReLU
BatchNorm
ReLU
shortcut
BPM
BPM
BatchNorm
Linear
TPM
gRPM
Tanh
TPM
gRPM
TPM
gRPM
(128)
(128*8, 4, 4)
(128*6, 4, 4)
(128*4, 8, 8)
(128*2, 16, 16)
(3, 32, 32)
(128*2, 8, 8)
(128*1, 16, 16)

62. 62
Linear
TPM
gRPM
Tanh
TPM
gRPM
TPM
gRPM
(128)
(128*8, 4, 4)
(128*8, 4, 4)
(128*4, 8, 8)
(128*2, 16, 16)
(3, 32, 32)
(128*4, 8, 8)
(128*1, 16, 16)
(128)
(128*8, 2, 2)
(128*8, 2, 2)
Linear
TPM
gRPM
TPM
gRPM
TPM
gRPM
Tanh
TPM
gRPM
(128*4, 4, 4)
(128*4, 4, 4)
(128*2, 8, 8)
(128*2, 8, 8)
(128*1, 16, 16)
(3, 32, 32)
(128*1, 16, 16)
(128)
(128*8, 1, 1)
(128*8, 1, 1)
Linear
TPM
gRPM
TPM
gRPM
TPM
gRPM
TPM
gRPM
(128*4, 4, 4)
(128*4, 4, 4)
(128*2, 8, 8)
(128*2, 8, 8)
(128*1, 16, 16)
(3, 32, 32)
(128*1, 16, 16)
Tanh
TPM
gRPM
(128*6, 2, 2)
(128*6, 2, 2)

63. 63
Linear
Trans-Convolution
Leakly ReLU
Trans-Convolution
BatchNorm
Leakly ReLU
Trans-Convolution
Tanh
BatchNorm
(128)
(128*8, 4, 4)
(128*4, 8, 8)
(3, 32, 32)
(128*1, 16, 16)

64. 64
dataset PGv1 vs CD PGv2 vs CD PGv3 vs CD CG vs CD
cifar10 6.0 (±0.5) 5.3 (±0.5) 4.3 (±0.3) 5.3 (±0.2)
LSUN 3.8 (±0.2) 3.8 (±0.2) 3.3 (±0.5) 3.2 (±0.2)
ImageNet 6.5 (±0.5) 6.5 (±0.3) 4.2 (±0.5) 4.5 (±0.3)

65. 65

66. 66

67. 67
(128*1, 16, 16)
(3, 32, 32)Perturbation
MaxPooling
BatchNorm
Linear Comb
ReLU
ReLU
BatchNorm
Linear Comb
ReLU
shortcut
BPM
BPM
dRPM
Convolution
BatchNorm
ReLU
MaxPooling
dRPM
dRPM
AvgPooling
Linear
(128*2, 8, 8)
(128*4, 4, 4)
(128*8, 2, 2)
(1)
Leakly ReLU
Tan
BatchNorm
Convolution
Convolution
Leakly ReLU
BatchNorm
Convolution
Leakly ReLU
Convolution
Sigmoid
BatchNorm
Leakly ReLU
Linear
AvgPooling
(128*8, 1, 1)
(128*1, 16, 16)
(3, 32, 32)
(128*2, 8, 8)
(128*4, 4, 4)
(128*8, 2, 2)
(1)
(128*8, 1, 1)

68. 68
(128*1, 16, 16)
(3, 32, 32)
dRPM
Convolution
BatchNorm
ReLU
MaxPooling
dRPM
dRPM
AvgPooling
Linear
(128*2, 8, 8)
(128*4, 4, 4)
(128*8, 2, 2)
(1)
Leakly ReLU
Tan
BatchNorm
Convolution
Convolution
Leakly ReLU
BatchNorm
Convolution
Leakly ReLU
Convolution
Sigmoid
BatchNorm
Leakly ReLU
Linear
AvgPooling
(128*8, 1, 1)
(128*1, 16, 16)
(3, 32, 32)
(128*2, 8, 8)
(128*4, 4, 4)
(128*8, 2, 2)
(1)
(128*8, 1, 1)

69. 69
dataset PGv1 vs. PD PGv1 vs. CD CG vs. CD
cifar10 5.6 (±0.3) 6.0 (±0.5) 5.3 (±0.2)
LSUN 3.6 (±0.2) 3.8 (±0.2) 3.2 (±0.2)
ImageNet 6.5 (±0.5) 6.5 (±0.5) 4.5 (±0.3)

70. 70

71. 71

72. 72

73. 73

74. 74
cifar10
WGAN-gp
[9]
Obake-GAN
(PGv1 vs PD)
iter 0 iter 1,000 iter 100,000

75. 75
ImageNet
WGAN-gp
[9]
Obake-GAN
(PGv1 vs PD)
iter 0 iter 1,000 iter 100,000

76. 76
Trainable
Parameters
2.Perturbation 3.Activation
Fixed Perturbation Masks
(non-trainable)
ReLU
Linear Weights
Output
4.Linear Combination1.Upsampling
(Bilinear)
Input

77. 77
2 – 1
11937
2 – 1
11937
2
31

78. 78
dataset MT(SND) MT(UD) LC(UD)
cifar10 5.6 (±0.3) 5.6 (±0.3) 4.5 (±1.0)
LSUN 3.6 (±0.2) 3.8 (±0.1) 3.2 (±0.4)
ImageNet 6.5 (±0.5) 5.1 (±0.9) 6.0 (±0.3)

79. 79

80. 80

81. 81

82. 82

83. 83

84. 84

85. 85

86. 86

87. 87

88. 88

89. 89
[1] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets.
In Advances in neural information processing systems (pp. 2672-2680).
[2] Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv
preprint arXiv:1710.10196.
[3] Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., & He, X. (2018). Attngan: Fine-grained text to image generation with
attentional generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp.
1316-1324).
[4] Brock, A., Donahue, J., & Simonyan, K. (2018). Large scale gan training for high fidelity natural image synthesis. arXiv preprint
arXiv:1809.11096.
[5] Y.Kishi, T.Ikegami, S.-i.O’uchi, R.Takano, W.Nogami, and T.Kudoh. “QuantizationOptimization for Training of Generative
Adversarial Network”, in 2018 Summer UnitedWorkshops on Parallel Distributed and Cooperative Processing (SWoPP). 2018-
ARC-23213, 7 2018, pp. 1-6
[6] Juefei-Xu, F., Naresh Boddeti, V., & Savvides, M. (2018). Perturbative neural networks. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition (pp. 3310-3318).
[7] Ali Farhadi, “Image Sampling” Retrieved from https://slideplayer.com/slide/6019094/ (Access:2019, January 27)
[8] Aljahdali, A., & Mascagni, M. (2017). Feistel-inspired scrambling improves the quality of linear congruential generators. Monte Carlo
Methods and Applications, 23(2), 89-99.
[9] Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. C. (2017). Improved training of wasserstein gans. In Advances in
Neural Information Processing Systems (pp. 5767-5777).