Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Digital Garage & Naviplus 主催
第2回Pythonで実践する深層学習
浅川伸一 asakawa@ieee.org
10/06/2016
2/24
Notice
次回以降で取り上げる話題,データ,プロジェクト,質疑応答のために
slack のチームを作成しました。今回参加されなかった方でも自由に参
加できます。チーム名は deeppython.slack.c...
10/06/2016
3/24
本日のお品書き
●
Basic knowledge of neurons
● Cross entropy
●
Feedforwards and feedbacks of neural networks
●
Try...
10/06/2016
4/24
前回の補足
最速で理解するには
1. 線形回帰 linear regression
2. ロジスティック回帰 logistic regression
3. 正則化 regularization
4. 多層パーセプ...
10/06/2016
5/24
Warning: Stealing Machine Learning Models via
Prediction APIs https://arxiv.org/abs/1609.02943
Machine lea...
10/06/2016
6/24
Prerequisites
●
Basics about Neural Networks
– How the brain actually works?
– How parallel computation wo...
10/06/2016
7/24
A schematic neuron
There are many neurotransimtters, but
we deal with those as positive/negative
weights a...
10/06/2016
8/24
Why computer vision is so difficult?
http://xkcd.com/1425/
10/06/2016
9/24
Cross entropy
10/06/2016
10/24
playground.tensorflow.org
10/06/2016
11/24
Convnet.js http://cs.stanford.edu/people/karpathy/convnetjs/
10/06/2016
12/24
TensorFlow Tips
●
Computation graph (see http://colah.github.io/posts/2015-08-Backprop/ , その翻訳記事
は http:/...
10/06/2016
13/24
A computation graph
http://deeplearning.net/software/theano/extending/graphstructures.html
10/06/2016
14/24
Sample code of TensorFlow
import tensorflow as tf
W = tf.get_variable(shape=[], name='W')
b = tf.get_vari...
10/06/2016
15/24
The difference between placeholder and variable
Since Tensor computations compose graphs then it's better...
10/06/2016
16/24
The difference between placeholder and variable
A placeholder is a handle of a value in the operation and...
10/06/2016
17/24
Convolutional neural networks: LeNet5
LeCun1998 より
10/06/2016
18/24
AlexNet
Krizensky+2012 より
10/06/2016
19/24
GoogLeNet
10/06/2016
20/24
GoogLeNet Inception module
10/06/2016
21/24
畳み込み演算
10/06/2016
22/24
kernels
10/06/2016
23/24
To understand convolution
●
畳み込みの理解には
http://colah.github.io/posts/2014-07-Understanding-Convolutions/
10/06/2016
24/24
What is convolution?
●
Convolution is an operation from signal processing
● Filters, or kernels in machin...
Upcoming SlideShare
Loading in …5
×

2016 dg2

530 views

Published on

第2回「TensorFlowで実践する深層学習」勉強会資料

Published in: Science
  • Be the first to comment

2016 dg2

  1. 1. Digital Garage & Naviplus 主催 第2回Pythonで実践する深層学習 浅川伸一 asakawa@ieee.org
  2. 2. 10/06/2016 2/24 Notice 次回以降で取り上げる話題,データ,プロジェクト,質疑応答のために slack のチームを作成しました。今回参加されなかった方でも自由に参 加できます。チーム名は deeppython.slack.com です。ご参加ください。 参加ご希望の方は deeplearning.w.python@gmail.com までメールをお 願いします。 #DLwPY
  3. 3. 10/06/2016 3/24 本日のお品書き ● Basic knowledge of neurons ● Cross entropy ● Feedforwards and feedbacks of neural networks ● Try googleplayground ● Convolutional neural networks ● Try keras
  4. 4. 10/06/2016 4/24 前回の補足 最速で理解するには 1. 線形回帰 linear regression 2. ロジスティック回帰 logistic regression 3. 正則化 regularization 4. 多層パーセプトロン multi-layered perceptrons 5. 畳み込みニューラルネットワーク convolutional neural networks 6. リカレントニューラルネットワーク recurrent neural networks 7. 強化学習 reinforcement learning
  5. 5. 10/06/2016 5/24 Warning: Stealing Machine Learning Models via Prediction APIs https://arxiv.org/abs/1609.02943 Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis. The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.
  6. 6. 10/06/2016 6/24 Prerequisites ● Basics about Neural Networks – How the brain actually works? – How parallel computation works adapting parameters inspired by neurons? – How the brain implements learning algorithms? ● (ペ)What is it a good idea to try to emulate the brain when solving a recognition task?
  7. 7. 10/06/2016 7/24 A schematic neuron There are many neurotransimtters, but we deal with those as positive/negative weights and also positive negative inputs. (ペ) why? http://www.mhhe.com/socscience/intro/ibank/set1.htm
  8. 8. 10/06/2016 8/24 Why computer vision is so difficult? http://xkcd.com/1425/
  9. 9. 10/06/2016 9/24 Cross entropy
  10. 10. 10/06/2016 10/24 playground.tensorflow.org
  11. 11. 10/06/2016 11/24 Convnet.js http://cs.stanford.edu/people/karpathy/convnetjs/
  12. 12. 10/06/2016 12/24 TensorFlow Tips ● Computation graph (see http://colah.github.io/posts/2015-08-Backprop/ , その翻訳記事 は http://postd.cc/2015-08-backprop/ ) ● Ways of installations (c.f. Tensorflow.org Download and Setup ) – Pip – Virutalenv – Anaconda – Docker ● Let’s try http://playground.tensorflow.org/ ● You can also check it out, Karpthy’s convnetjs ● Keras is another choice to consider
  13. 13. 10/06/2016 13/24 A computation graph http://deeplearning.net/software/theano/extending/graphstructures.html
  14. 14. 10/06/2016 14/24 Sample code of TensorFlow import tensorflow as tf W = tf.get_variable(shape=[], name='W') b = tf.get_variable(shape=[], name='b') x = tf.placeholder(shape=[None], dtype=tf.float32, name='x') y = tf.matmul(W, x) + b with tf.Session() as sess: sess.run(tf.initialize_all_variables()) print(sess.run(y, feed_dict={x: x_in}))
  15. 15. 10/06/2016 15/24 The difference between placeholder and variable Since Tensor computations compose graphs then it's better to interpret the two in terms of graphs. When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. Then during the process of the an operation variables should be constant. import tensorflow as tf # Create a variable. # w = tf.Variable(<initial-value>, name=<optional-name>) w = tf.Variable(tf.truncated_normal([10, 40])) v = tf.Variable(tf.truncated_normal([40, 20])) # Use the variable in the graph like any Tensor. # The variable should be initialized before this operation! y = tf.matmul(w, v) # Assign a new value to the variable with `assign()` or a related method. w.assign(w + 1.0) w.assign_add(1.0) http://stackoverflow.com/questions/36693740/whats-the-difference-between-tf-placeholder-and-tf-variable tf.Variableはオペレーション実行前に初期化される
  16. 16. 10/06/2016 16/24 The difference between placeholder and variable A placeholder is a handle of a value in the operation and it can be not initialized before the execution of the graph (launching the graph in the session which does its computation relaying on a highly efficient C+ + backend). x = tf.placeholder(tf.float32, shape=(1024, 1024)) # You don't need to initialize it to calculate y, it's different from # the variable above, the placeholder is a "variable"(not intialized) # in this operation. y = tf.matmul(x, x) with tf.Session() as sess: # However you should initialize x to execute y for the execution of the graph. print(sess.run(y)) # ERROR: will fail because x was not fed. rand_array = np.random.rand(1024, 1024) print(sess.run(y, feed_dict={x: rand_array})) # Will succeed. プレースホルダーは初期化されない
  17. 17. 10/06/2016 17/24 Convolutional neural networks: LeNet5 LeCun1998 より
  18. 18. 10/06/2016 18/24 AlexNet Krizensky+2012 より
  19. 19. 10/06/2016 19/24 GoogLeNet
  20. 20. 10/06/2016 20/24 GoogLeNet Inception module
  21. 21. 10/06/2016 21/24 畳み込み演算
  22. 22. 10/06/2016 22/24 kernels
  23. 23. 10/06/2016 23/24 To understand convolution ● 畳み込みの理解には http://colah.github.io/posts/2014-07-Understanding-Convolutions/
  24. 24. 10/06/2016 24/24 What is convolution? ● Convolution is an operation from signal processing ● Filters, or kernels in machine learning,

×