Recommended
More Related Content
Similar to 2016 dg2
Similar to 2016 dg2 (20)
More from Shin Asakawa
Recently uploaded
Recently uploaded (20)
2016 dg2
- 1. Digital Garage & Naviplus 主催 第２回Pythonで実践する深層学習 浅川伸一 asakawa@ieee.org
- 2. 10/06/2016 2/24 Notice 次回以降で取り上げる話題，データ，プロジェクト，質疑応答のために slack のチームを作成しました。今回参加されなかった方でも自由に参 加できます。チーム名は deeppython.slack.com です。ご参加ください。 参加ご希望の方は deeplearning.w.python@gmail.com までメールをお 願いします。 ＃DLｗPY
- 3. 10/06/2016 3/24 本日のお品書き ● Basic knowledge of neurons ● Cross entropy ● Feedforwards and feedbacks of neural networks ● Try googleplayground ● Convolutional neural networks ● Try keras
- 4. 10/06/2016 4/24 前回の補足 最速で理解するには 1. 線形回帰 linear regression 2. ロジスティック回帰 logistic regression 3. 正則化 regularization 4. 多層パーセプトロン multi-layered perceptrons 5. 畳み込みニューラルネットワーク convolutional neural networks 6. リカレントニューラルネットワーク recurrent neural networks 7. 強化学習 reinforcement learning
- 5. 10/06/2016 5/24 Warning: Stealing Machine Learning Models via Prediction APIs https://arxiv.org/abs/1609.02943 Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis. The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.
- 6. 10/06/2016 6/24 Prerequisites ● Basics about Neural Networks – How the brain actually works? – How parallel computation works adapting parameters inspired by neurons? – How the brain implements learning algorithms? ● (ペ)What is it a good idea to try to emulate the brain when solving a recognition task?
- 7. 10/06/2016 7/24 A schematic neuron There are many neurotransimtters, but we deal with those as positive/negative weights and also positive negative inputs. (ペ) why? http://www.mhhe.com/socscience/intro/ibank/set1.htm
- 8. 10/06/2016 8/24 Why computer vision is so difficult? http://xkcd.com/1425/
- 12. 10/06/2016 12/24 TensorFlow Tips ● Computation graph (see http://colah.github.io/posts/2015-08-Backprop/ , その翻訳記事 は http://postd.cc/2015-08-backprop/ ) ● Ways of installations (c.f. Tensorflow.org Download and Setup ) – Pip – Virutalenv – Anaconda – Docker ● Let’s try http://playground.tensorflow.org/ ● You can also check it out, Karpthy’s convnetjs ● Keras is another choice to consider
- 14. 10/06/2016 14/24 Sample code of TensorFlow import tensorflow as tf W = tf.get_variable(shape=[], name='W') b = tf.get_variable(shape=[], name='b') x = tf.placeholder(shape=[None], dtype=tf.float32, name='x') y = tf.matmul(W, x) + b with tf.Session() as sess: sess.run(tf.initialize_all_variables()) print(sess.run(y, feed_dict={x: x_in}))
- 15. 10/06/2016 15/24 The difference between placeholder and variable Since Tensor computations compose graphs then it's better to interpret the two in terms of graphs. When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. Then during the process of the an operation variables should be constant. import tensorflow as tf # Create a variable. # w = tf.Variable(<initial-value>, name=<optional-name>) w = tf.Variable(tf.truncated_normal([10, 40])) v = tf.Variable(tf.truncated_normal([40, 20])) # Use the variable in the graph like any Tensor. # The variable should be initialized before this operation! y = tf.matmul(w, v) # Assign a new value to the variable with `assign()` or a related method. w.assign(w + 1.0) w.assign_add(1.0) http://stackoverflow.com/questions/36693740/whats-the-difference-between-tf-placeholder-and-tf-variable tf.Variableはオペレーション実行前に初期化される
- 16. 10/06/2016 16/24 The difference between placeholder and variable A placeholder is a handle of a value in the operation and it can be not initialized before the execution of the graph (launching the graph in the session which does its computation relaying on a highly efficient C+ + backend). x = tf.placeholder(tf.float32, shape=(1024, 1024)) # You don't need to initialize it to calculate y, it's different from # the variable above, the placeholder is a "variable"(not intialized) # in this operation. y = tf.matmul(x, x) with tf.Session() as sess: # However you should initialize x to execute y for the execution of the graph. print(sess.run(y)) # ERROR: will fail because x was not fed. rand_array = np.random.rand(1024, 1024) print(sess.run(y, feed_dict={x: rand_array})) # Will succeed. プレースホルダーは初期化されない
- 17. 10/06/2016 17/24 Convolutional neural networks: LeNet5 LeCun1998 より
- 24. 10/06/2016 24/24 What is convolution? ● Convolution is an operation from signal processing ● Filters, or kernels in machine learning,