Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep LearningフレームワークChainerと最近の技術動向

8,806 views

Published on

日本ソフトウェア科学会第34回大会 2017 チュートリアル
#chainer #chainercv #chainerrl #deeplearning #セグメンテーション #ニューラルネットワーク #強化学習 #深層学習 #物体検出 #画像認識 #自然言語処理

Published in: Software
  • Are you financially down or do you need loan to setup your own business or taking your business to a higher level? We give out loan as low at 2% rate will offer business loan home loan, loan for real estate business any kind of loan Interested,Contact us via email: maggetfundslimited@mail.com for more information
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Deep LearningフレームワークChainerと最近の技術動向

  1. 1. Deep Learning Chainer 34 2017 Researcher @ Preferred Networks 1
  2. 2. Agenda • • • • • • • Chainer • numpy Chainer • • • GAN • • 2
  3. 3. • • • 2010 • 2012 • 2013 - 2014 UC Berkeley (Visiting Student Researcher) • 2015 • 2016 @ "Semantic Segmentation for Aerial Imagery with Convolutional Neural Network" • 2016.9 Facebook, Inc. • 2016.9 Preferred Networks, Inc. 3
  4. 4. • Preferred Networks • PFN • 106 8 • • FANUC Toyota NTT • We are hiring! 4
  5. 5. PFN AutomotiveHumanoid Robot Consumer Industrial Cloud Device Photo Game Text Speech Infrastructure Factory Robot Automotive Healthcare Smart City Industry4.0 Industrial IoT 5
  6. 6. • • Chainer Define-by-Run • Chainer • GAN • ChaineCV ChainerRL 6
  7. 7. 7
  8. 8. 8
  9. 9. • • fully-connected layer 9
  10. 10. 4 3 2 1 0 1 2 3 4 x 1 0 1 2 3 4 y y = 1/(1 + exp(x)) y = tanh(x) y = max(0, x) • ReLU • activation function Maxout1 , LReLU2 , PReLU3 , ELU4 , SELU5 , etc. 10
  11. 11. "core idea" The core idea in deep learning is that we assume that the data was generated by the composition of factors or features, potentially at multiple levels in a hierarchy. — Ian Goodfellow, Yoshua Bengio, Aaron Courville, "Deep Learning" 11
  12. 12. . representation 12
  13. 13. h(1) = a(1) (W(1) x + b(1) ) h(l) = a(l) (W(l) h(l 1) + b(l 1) ) o = a(o) (W(o) h(o) + b(o) ) • • • hidden layer 1 1 1 Kurt Hornik, Maxwell Stinchcombe, Halbert White, "Multilayer feedforward networks are universal approximators", Neural Networks (1989) 13
  14. 14. A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html (1) • • 1 14
  15. 15. A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html (2) • 2 • 15
  16. 16. A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html (3) • • • 16
  17. 17. A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html (4) • • 17
  18. 18. A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html (5) • • • 4 18
  19. 19. (6) • 6 • 2 6 “A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html 19
  20. 20. (7) • • 7 2 7 “A visual proof that neural nets can compute any function” http://neuralnetworksanddeeplearning.com/chap4.html 20
  21. 21. so many nodes Replacable? • 1 • 21
  22. 22. • • … 8 • piecewise linear function 9 9 Razvan Pascanu, Guido Montufar, and Yoshua Bengio, "On the number of response regions of deep feed forward networks with piece-wise linear activations", NIPS (2014) 8 Merrick Furst, James B. Saxe, and Michael Sipser, "Parity, Circuits, and the Polynomial-Time Hierarchy", Mathematical systems theory (1984) 22
  23. 23. • training data • • 23
  24. 24. • • loss function • 24
  25. 25. • • softmax • one-hot • cross entropy 25
  26. 26. • gradient method gradient descent • • learning rate • 26
  27. 27. backpropagation • • • 27
  28. 28. • minibatch • An overview of gradient descent optimization algorithms 28
  29. 29. SGD without momentum SGD with momentum * Stochastic gradient descent (SGD) Momentum SGD15 15 A. Krizhevsky, I. Sutskever, G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NIPS (2012) 29
  30. 30. Nesterov accelerated gradient (NAG)16 Momentum SGD RNN AdaGrad17 AdaDelta18 AdaGrad RMSProp19 , Adam20 , Eve21 … 30
  31. 31. : sigmoid cross entropy • • 31
  32. 32. : sigmoid cross entropy • exploding gradients …↓ • vanishing gradient 32
  33. 33. 33
  34. 34. Greedy layer-wise pre-training10 • Yoshua Bengio ICML 2009 10 Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, "Greedy layer-wise training of deep networks", NIPS (2006) 34
  35. 35. ReLU rectified linear unit 11 • Sigmoid 0 11 Xavier Glorot, Antoine Bordes, and Yoshua Bengio, "Deep Sparse Rectifier Neural Networks", NIPS Workshop (2010) 35
  36. 36. Dropout12 • [13] Figure 1 13 13 N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A Simple Way to Prevent Neural Networks from Overfitting", Journal of Machine Learning Research (2014) 12 G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors", On arxiv (2012) 36
  37. 37. [14] Residual learning 14 • • 14 K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition", CVPR (2016) 37
  38. 38. 38
  39. 39. • 1. forward Dropout 2. backward 3. autograd 2. 4. optimizer 3. 39
  40. 40. Chainer Preferred Networks Python https://github.com/chainer/chainer 40
  41. 41. Popularity Growth of Chainer 41
  42. 42. Define-and-Run Define-by-Run Define-and-Run Define-and-Run Define-by-Run Define-and-Run Chainer • • Define-and-Run • Define-by-Run • Define-and-Run 42
  43. 43. Define-and-Run Define-by-Run 43
  44. 44. Define-by-Run • • Python • Caffe prototxt • recurrent neural network; RNN for BPTT backpropagation through time 44
  45. 45. numpy Chainer 45
  46. 46. Variable Function Function Function Variable Variable VariableVariable Variable VariableVariable Variable (1) Variable Function • 2 Variable Function • Variable Function directed asyclic graph; DAG • Variable Function rank 46
  47. 47. rank=0 rank=1 rank=2 Variable Function Function Function Variable Variable VariableVariable Variable VariableVariable Variable creator creator inputs inputs inputs (2) • backward • Function Variable inputs • Function Variable Function creator • Function Variable rank Function Variable Function 1 rank 47
  48. 48. Chainer Chainer 48
  49. 49. grad_outputs .backward(inputs, grad_outputs) .backward(inputs, grad_outputs) grad Function Function .backward() grad Variable .inputs Variable Variable grad_outputs .creator.creator grad_outputs grad grad grad VariableVariable Variable .backward(inputs, grad_outputs) grad_outputs grad grad grad VariableVariable Variable Function .inputs creator .inputs =1 creator (1) • Define-by-Run backward Variable • Variable backward() Function backward() • inputs • grad_outputs 49
  50. 50. (2) • Variable Chainer • • grad 50
  51. 51. (3) • Function backward() • Function outputs outputs→creator→outputs • Function Function x, W, b = Variable(init_x), Variable(init_W), Variable(init_b) y = LinearFunction()(x, W, b) # forward # ...backward, update ... y = LinearFunction()(x, W, b) # forward 51
  52. 52. Link • W, b Function • Link • params() 52
  53. 53. Chain • • • Link params() Link 53
  54. 54. Optimizer • Optimizer setup() Chain Link update() • Optimizer • Chainer state 54
  55. 55. • Linear ReLUFunction • ReLU ReLUFunction • ReLU ( 55
  56. 56. • 2 mean squared error; MSE • MSE ReLU 56
  57. 57. • : MNIST • 100 1 forward • SGD Optimizer setup() 57
  58. 58. • MNIST scikit-learn from sklearn.datasets import fetch_mldata mnist = fetch_mldata('MNIST original', data_home='./') x, t = mnist['data'] / 255, mnist['target'] t = numpy.array([t == i for i in range(10)]).T train_x, train_t = x[:60000], t[:60000] val_x, val_t = x[60000:], t[60000:] • 1 • 150 58
  59. 59. • 94% • http://bit.ly/mini_chainer_mnist • numpy Define-by-Run 59
  60. 60. Chainer 60
  61. 61. Trainer (1) Optimizer • Trainer • Chainer Chain • chainer.optimizer SGD, MomentumSGD, NesterovAG, RMSprop, RMSpropGraves, AdaGrad, AdaDelta, Adam, etc... 61
  62. 62. Trainer (2) • Chainer • MNIST • Validation len(val) 62
  63. 63. Trainer (3) • • Chainer chainer.functions chainer.links • softmax cross entropy 63
  64. 64. Trainer • Trainer Optimizer • • Trainer Extension import import 64
  65. 65. Trainer (1) • Extension • snapshot • LogReport, PrintReport • validation Evaluator • PlotReport • Graphviz dot dump_graph • ParameterStatistics • • Trainer extensions (https://docs.chainer.org/en/stable/reference/extensions.html) 65
  66. 66. Trainer (2) • • • Trainer • GPU ParallelUpdater, MultiprocessParallelUpdater MultiprocessIterator 66
  67. 67. GPU • • Chainer CuPy GPGPU • CuPy NCCL NCCL2 GPU • cuDNN NVIDIA v7 CUDA v9 • fp16 67
  68. 68. CuPy • CuPy NumPy NVIDIA CUDA GPU • NumPy API NumPy GPU • GPU • KMeans, Gaussian Mixture Model Example CuPy: https://github.com/cupy/cupy 68
  69. 69. 69
  70. 70. • • • Semantic Segmentation • Instanse-aware Segmentation 70
  71. 71. horse : 94% dog : 3% pig : 2% cat : 1% . . . 71
  72. 72. R G B 72
  73. 73. • • convolutional layer • pooling [A. Krizhevsky, 2016]15 ** 15 A. Krizhevsky, I. Sutskever, G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NIPS (2012) 73
  74. 74. • convolutional neural network; CNN • • 1 74
  75. 75. • • • stride • padding CS231n Convolutional Neural Networks for Visual Recognition 75
  76. 76. • • receptive field • CS231n Convolutional Neural Networks for Visual Recognition 76
  77. 77. 77
  78. 78. 2012 Toronto Geoffry Hinton 2 10% 28.191 25.77 15.315 11.743 6.656 5.1 3.567 2.991 2.251 ILSVRC Object Classification (Top-5 Error)ImageNet Large Scale Visual Recognition Challenge (ILSVRC) • 1000 1 128 1000 • 2010 • 2011 localization • 2012 Fine-grained • 2013 bounding box • 2015 • 2016 78
  79. 79. AlexNet15 • 2012 ILSVRC • 224x244 5 3 • LRN (local response normalization) ReLU max pooling • AlexNet ImageNet pre-trained model • AlexNet pre-trained model transfer learning 15 A. Krizhevsky, I. Sutskever, G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NIPS (2012) 79
  80. 80. GoogleNet16 • 2014 ILSVRC Inception ResNet • Inception 22 • 1x1, 3x3, 5x5 concat • 1x1 16 C. Szegedy, et. al., "Going Deeper with Convolutions", CVPR (2015) 80
  81. 81. VGG17 • 2014 GoogLeNet ILSVRC 2 • 3x3 1 receptive field 3x3 2 5x5, 3 7x7 receptive field • ResNet pre-trained 17 K. Simonyan, A. Zisserman "Very Deep Convolutional Networks for Large-Scale Image Recognition" arXiv technical report, (2014) 81
  82. 82. ResNet18 • ILSVRC 2015 GoogLeNet 22 152 CIFAR 1502 • • 18 Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun , "Deep Residual Learning for Image Recognition." arXiv:1512.03385 (2015) 82
  83. 83. Wide-ResNet19 • ResNet " " ResNet • Residual block Dropout ratio 30~40% 19 Sergey Zagoruyko, Nikos Komodakis, "Wide Residual Networks", arXiv: 1605.07146 (2016) 83
  84. 84. DenseNet20 • CVPR 2017 Residual connection 2 ResBlock • • 20 Gao Huang, Zhuang Liu, Kilian Q. Weinberger, Laurens van der Maaten, "Densely Connected Convolutional Networks", CVPR (2017) 84
  85. 85. ResNeXt21 • 2016 ILSVRC 2 • • cardinality • cardinality 21 Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, Kaiming He, "Aggregated Residual Transformations for Deep Neural Networks", CVPR (2017) 85
  86. 86. Squeeze-and-Excitation22 • 2017 ILSVRC • Inception • • squeeze excitation 22 Jie Hu, Li Shen, Gang Sun, "Squeeze-and-Excitation Networks", arXiv pre-print: 1709.01507 (2017) 86
  87. 87. - 2017 ILSVRC Kaggle ImageNet Object Localization Challenge) - - MSCOCO Dataset - Places Challenge (ADE20K) - YouTube-8M Video Understanding Challenge - Cityscapes Dataset - Mapillary Vistas Datset - VisualGenome 87
  88. 88. 88
  89. 89. Fast R-CNN Faster R-CNN Faster/Fast R-CNN/R-CNN23 • R-CNN: region proposals selective search CNN resize CNN SVM • Fast R-CNN: RoI Pooling bounding box bbox regression • Faster R-CNN: Region proposal network (RPN) CNN RPN / bbox 23 Ren, Shaoqing, et al. "Faster R-CNN: Towards real-time object detection with region proposal networks." NIPS (2015) 89
  90. 90. You Only Look Once (YOLO)25 • bounding box • bounding box 1 end-to-end • FCN YOLO9000 CVPR2017 25 Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." arXiv preprint arXiv:1506.02640 (2015) 90
  91. 91. Single Shot Multibox Detector (SSD)24 • YOLO Faster R-CNN • • • End-to-end 24 Liu, Wei, et al. "SSD: Single Shot MultiBox Detector." ECCV (2016) 91
  92. 92. Semantic Segmentation 92
  93. 93. Semantic Segmentation • • " " • Instance-aware 93
  94. 94. (1) Dilated convolution51 • • 51 "Multi-Scale Context Aggregation by Dilated Convolutions", ICLR 2016 94
  95. 95. (2) Multi-scale feature ensemble • • 52 52 "Hypercolumns for Object Segmentation and Fine-grained Localization", CVPR 2015 95
  96. 96. (3) Conditional random field (CRF) • CNN refine (DeepLab53 ) • DeepLab refine End-to-End (DPN54 , CRF as RNN55 , Detections and Superpixels56 ) 56 "Higher order conditional random fields in deep neural networks", ECCV 2016 55 "Conditional random fields as recurrent neural networks", ICCV 2015 54 "Semantic image segmentation via deep parsing network", ICCV 2015 53 "Semantic image segmentation with deep convolutional nets and fully connected crfs", ICLR 2015 96
  97. 97. (4) Global average pooling (GAP) • • ParseNet57 Global average pooling FCN 57 "Parsenet: Looking wider to see better", ICLR 2016 97
  98. 98. (1) Mismatched relationship • • • • FCN 98
  99. 99. (2) Confusing Classes • • ADE20K 17.6% 58 • FCN • 58 "Semantic understanding of scenes through the ADE20K dataset", CVPR 2017 99
  100. 100. (3) Inconspicuous Classes • • FCN • sub- region 100
  101. 101. • • • 101
  102. 102. Fully Convolutional Network26 • Classification pre-training 1x1 • Deconvolution • semantic low level skip connection 26 Jonathan Long and Evan Shelhamer et al., "Fully Convolutional Networks for Semantic Segmentation", appeared in arxiv on Nov. 14, 2014 102
  103. 103. Deconvolution • Deconvolution transposed convolution backward convolution • Convolution 1. stride 2. -1 3. padding 4. Convolution • stride Convolution arithmetic https://github.com/vdumoulin/ conv_arithmetic 103
  104. 104. Global Average Pooling (GAP)59 • • ResNet receptive field • GAP • 59 "Parsenet: Looking wider to see better", ICLR 2016 104
  105. 105. SegNet27 • • Max pooling • 0 27 Vijay Badrinarayanan, Alex Kendall and Roberto Cipolla "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation." PAMI, (2017) 105
  106. 106. U-Net28 • • Max pooling Deconvolution • concat • "U" U- Net 28 “U-Net: Convolutional Networks for Biomedical Image Segmentation”, Olaf Ronneberger, Philipp Fischer, Thomas Brox, 18 May 2015 106
  107. 107. PSPNet60 Pyramid Pooling Module 60 Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia, "Pyramid Scene Parsing Network", CVPR (2017) 107
  108. 108. Pose Estimation 108
  109. 109. Pose Affinity Field33 • CNN Convolutional Pose Machine Part Affinity Field CNN • OpenPose https://github.com/CMU-Perceptual-Computing- Lab/openpose • Geforce GTX 1080 9fps 33 Zhe Cao and Tomas Simon and Shih-En Wei and Yaser Sheikh, "Realtime Multi-Person Pose Estimation using Part Affinity Fields", CVPR (2017) 109
  110. 110. • → Faster R-CNN • → 110
  111. 111. repeatability reproducibility repeatability reproducibility repeatability reproducibility ACM Repeatability (same team, same experimental setup), Replicability (different team, same experimental setup), Reproducibility (different team, different experiental setup) http://www.acm.org/publications/policies/artifact-review-badging 111
  112. 112. ChainerCV https://github.com/chainer/chainercv ChainerCV • • • • pre-trained 112
  113. 113. ChainerCV 113
  114. 114. - Faster R-CNN (VGG16-based, ) - SSD300, SSD512 : - SegNet - PSPNet (coming soon) - PASCAL VOC (bounding box, segmentation) - Stanford Online Products (classification) - CamVid (segmentation) - Caltech-UCSD Birds-200 (classification, key-points) - Cityscapes (segmentation) 114
  115. 115. : pretrained-model ChainerCV pretarined_model model = FasterRCNNVGG16() # # PASCAL VOC2007 model = FasterRCNNVGG16(pretrained_model='voc07') model = SSD300(pretrained_model='voc0712') model = SegNet(pretrained_model='camvid') 115
  116. 116. predict() ChainerCV predict() # bboxes, labels, scores = model.predict(imgs) # labels = model.predict(imgs) 116
  117. 117. predict() 1. 2. forward 3. non-maximum supression 117
  118. 118. [37] D. Xu, Y. Zhu, C. B. Choy, L. Fei-Fei, “Scene Graph Generation by Iterative Message Passing”, CVPR (2017) • Faster R-CNN Region Proposal network (RPN) • RPN … • 118
  119. 119. ChainerCV • Chainer • public • from chainercv.datasets import VOCDetectionDataset dataset = VOCDetectionDatset(split='trainval', year='2007') # "trainval" 34 img, bbox, label = dataset[34] 119
  120. 120. ChainerCV transforms • data augmentation • ChainerCV data augmentation • • center_crop, pca_lighting, random_crop, random_expand, random_flip, random_rotate, ten_crop, etc... 120
  121. 121. ChainerCV transforms TransformDataset Chainer from chainercv import transforms def transform(in_data): img, bbox, label = in_data img, param = transforms.random_flip(img, x_flip=True, return_param=True) bbox = transforms.flip_bbox(bbox, x_flip=param['x_flip']) return img, bbox, label dataset = TransformDataset(dataset, transform) bounding box 121
  122. 122. bounding box ChainerCV • • • bounding box • matplotlib 122
  123. 123. ChainerCV mean Intersection over Union (mIoU) mean Average Precision (mAP) Chainer Trainer Extension # mAP Trainer Extension evaluator = chainercv.extension.DetectionVOCEvaluator(iterator, model) # # e.g., result['main/map'] result = evaluator() 123
  124. 124. • https://github.com/chainer/chainercv Faster R-CNN SegNet 124
  125. 125. GAN 125
  126. 126. • • • RBM restricted boltzmann machine 34 Variational Auto-Encoder (VAE)35 Generative Adversarial Nets (GAN) 35 Diederik P Kingma and Max Welling, "Auto-Encoding Variational Bayes", ICLR (2014) 34 Smolensky, Paul, "Chapter 6: Information Processing in Dynamical Systems: Foundations of Harmony Theory", Parallel Distributed Processing: Explorations in the Microstructure of Cognition (1986) 126
  127. 127. Generator Discriminator Dataset OR from dataset? Generative Adversarial Nets36 • unsupervised learning • G D • D G • G D 36 I. J. Goodfellowm, et. al., "Generative Adversarial Nets", NIPS (2014) 127
  128. 128. GAN • • Discriminator • Generator Discriminator 128
  129. 129. GAN • Generator • Generator 129
  130. 130. DCGAN37 • GAN • Generator 1 Deconvolution • Discriminator Generator • • 37 Alec Radford, Luke Metz, Soumith Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", ICLR (2016) 130
  131. 131. DCGAN37 • GAN DCGAN • D stride=2 • D Global Average Pooling • D Leaky ReLU • G D Batch Normalization G D 37 Alec Radford, Luke Metz, Soumith Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", ICLR (2016) 131
  132. 132. Improved Techniques for Training GANs38 • GAN • Feature matching: D fake real • Minibatch discrimination: G D 1 mode cllapse D concat • 38 T. Salimans, I. Goodfellow, et. al., "Improved Techniques for Training GANs", NIPS (2016) 132
  133. 133. Improved Techniques for Training GANs38 • Generator Semi-supervised learning • ImageNet DCGAN • Inception score GAN pre-trained model 38 T. Salimans, I. Goodfellow, et. al., "Improved Techniques for Training GANs", NIPS (2016) 133
  134. 134. Wasserstein GAN (WGAN)39 (1) • GAN Generator Wasserstein Earth Mover's Distance WGAN • Generator 2 Wasserstein 39 Martin Arjovsky, Soumith Chintala, Léon Bottou, "Wasserstein GAN", arXiv:1701.07875 (2017) 134
  135. 135. Wasserstein GAN (WGAN)39 (2) • WGAN Discriminator Wasserstein • Discriminator(D) D Wasserstein • 39 Martin Arjovsky, Soumith Chintala, Léon Bottou, "Wasserstein GAN", arXiv:1701.07875 (2017) 135
  136. 136. Wasserstein GAN (WGAN)39 (3) • WGAN Discriminator Wasserstein Wasserstein • Generator Wasserstein • Generator Wasserstein 39 Martin Arjovsky, Soumith Chintala, Léon Bottou, "Wasserstein GAN", arXiv:1701.07875 (2017) 136
  137. 137. Wasserstein GAN (WGAN)39 (4) 1. Discriminator 2. 0.01 3. Generator 4. 39 Martin Arjovsky, Soumith Chintala, Léon Bottou, "Wasserstein GAN", arXiv:1701.07875 (2017) 137
  138. 138. WGAN with Gradient Penalty (WGAN-GP) 40 • WGAN Discriminator Gradient Penalty • Chainer v3 40 Gulrajani, Ishaan, et al. "Improved training of wasserstein gans." arXiv preprint arXiv:1704.00028 (2017). 138
  139. 139. Temporal Generative Adversarial Nets (TGAN)41 • WGAN • Video Generator Image Generator 41 Masaki Saito, Eiichi Matsumoto, Shunta Saito, "Temporal Generative Adversarial Nets with Singular Value Clipping", ICCV (2017) 139
  140. 140. Temporal Generative Adversarial Nets (TGAN)41 • GAN WGAN 1 singular value clipping • Inception score 41 Masaki Saito, Eiichi Matsumoto, Shunta Saito, "Temporal Generative Adversarial Nets with Singular Value Clipping", ICCV (2017) 140
  141. 141. SimGAN42 • CG Refiner • Refiner Discriminator adversarial loss self-regularization • Apple, inc. CVPR 2017 "Improving the Realism of Synthetic Images" 42 A. Shrivastava, et. al. "Learning from Simulated and Unsupervised Images through Adversarial Training", CVPR (2017) 141
  142. 142. Chainer-GAN-lib GAN Chainer Chainer Trainer GAN https://github.com/pfnet-research/chainer-gan-lib 142
  143. 143. Adversarial examples70 • " " • NIPS 2017: Non-targeted Adversarial Attack Google Brain "Non- targeted" 70 Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy, "Explaining and Harnessing Adversarial Examples." ICLR (2015) 143
  144. 144. 144
  145. 145. NLP • • part-of-speech tagging • word segmentation • word sense disambiguation • named entity extraction • syntactic parsing • predicate-argument recognition 145
  146. 146. NLP • • • 146
  147. 147. 147
  148. 148. RNN • RNN (recurrent neural networks) • RNN 148
  149. 149. http://qiita.com/t_Signull/items/21b82be280b46f467d1b LSTM • • 1 3 • : • 1 • • 149
  150. 150. 2.11 Gated recurrent unit (GRU) • LSTM • reset 1 • update) 1 • GRU LSTM 150
  151. 151. RNN one-hot RNN RNN 151
  152. 152. s.t. perplexity; PPL 2 PPL PPL 152
  153. 153. 1. Penn Treebank ptb 90 1 Chainer example ptb example https://github.com/chainer/chainer/ tree/master/examples/ptb 2. One Biliion Word 8 80 3. Hutter 90MB/5MB/5MB train/val/test 153
  154. 154. sequence-to-sequence • • RNN • • • 154
  155. 155. 155
  156. 156. greedy algorithm 156
  157. 157. 157
  158. 158. attention mechanism LSTM 158
  159. 159. Attention is all you need65 RNN/CNN Attention Transformer SOTA Transformer: A Novel Neural Network Architecture for Language Understanding 65 Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, "Attention Is All You Need", (2017) 159
  160. 160. 160
  161. 161. policy 1. 2. 3. 4. 161
  162. 162. • → return • discounted total reward • state value 162
  163. 163. state value function optimal policy 163
  164. 164. action value function 164
  165. 165. greedy ε-greedy greedy ε-greedy ε greedy 165
  166. 166. Q Q-learning Q Q 166
  167. 167. Q (1) Q Q 167
  168. 168. Q (2) Q Q →MSE SGD 168
  169. 169. Deep Q-Network (DQN)43 • Q • Q NN DQN • Experience replay • Target Q-Network • clipping DQN SPACE INVADERS (uploaded by DeepMind) 43 Mnih, Volodymyr, et al. "Playing Atari with Deep Reinforcement Learning", NIPS (2013) 169
  170. 170. DQN Experience replay - replay memory - Q Target Q-Network - Q - Q $theta$ Q clipping - clip 170
  171. 171. • Q • • 171
  172. 172. • 172
  173. 173. • 45 • 45 Pierre Andry, et. al., "Learning invariant sensorimotor behaviors: A developmental approach to imitation mechanisms." Adaptive behavior (2004) 173
  174. 174. Actor-Critic • Actor Critic • • 174
  175. 175. Asynchronous Advantage Actor-Critic (A3C)44 • • Actor-Critic • Experience replay RNN • 44 V. Mnih, et. al., "Asynchronous Methods for Deep Reinforcement Learning", ICML (2016) 175
  176. 176. DDPG46 • Deep Deterministic Policy Gradient (DDPG) Actor-Critic • Deep Q-Network End-to-End • Deep Reinforcement Learning (DDPG) demonstration 46 Lillicrap, Timothy P., et al. "Continuous control with deep reinforcement learning." arXiv preprint arXiv:1509.02971 (2015) 176
  177. 177. ChainerRL • Chainer https://github.com/chainer/chainerrl 177
  178. 178. ChainerRL : • ChainerRL OpenAI Gym Gym • reset step env = YourEnv() # reset obs = env.reset() action = 0 # step # 4 obs, r, done, info = env.step(action) 178
  179. 179. ChainerRL : (1) • • Chainer Q class CustomDiscreteQFunction(chainer.Chain): def __init__(self): super().__init__(l1=L.Linear(100, 50) l2=L.Linear(50, 4)) def __call__(self, x, test=False): h = F.relu(self.l1(x)) h = self.l2(h) return chainerrl.action_value.DiscreteActionValue(h) 179
  180. 180. ChainerRL : (2) class CustomGaussianPolicy(chainer.Chain): def __init__(self): super().__init__(l1=L.Linear(100, 50) mean=L.Linear(50, 4), var=L.Linear(50, 4)) def __call__(self, x, test=False): h = F.relu(self.l1(x)) mean = self.mean(h) var = self.var(h) return chainerrl.distribution.GaussianDistribution(mean, var) 180
  181. 181. ChainerRL : Q Chainer Optimizer q_func = CustomDiscreteQFunction() optimizer = chainer.Adam() optimizer.setup(q_func) agent = chainerrl.agents.DQN(q_func, optimizer, ...) # 181
  182. 182. ChainerRL : (1) ChainerRL • ChainerRL chainerrl.experiments.train_agent_with_evaluation( agent, env, steps=100000, eval_frequency=10000, eval_n_runs=10, outdir='results') 182
  183. 183. ChainerRL : (1) obs = env.reset() r = 0 done = False for _ in range(10000): while not done: action = agent.act_and_train(obs, r) obs, r, done, info = env.step(action) agent.stop_episode_and_train(obs, r, done) obs = env.reset() r, done = 0, False agent.save('final_agent') 183
  184. 184. ChainerRL Quick Start Guide https://github.com/chainer/chainerrl/ blob/master/examples/quickstart/ quickstart.ipynb OpenAI Gym DQN 184
  185. 185. Chainer 185

×