Download as PDF, PPTX
![Example : Binary Classification
n
w1
w2
1
b
y
x1
x2
x1
x2
y
y =
1
1 + e x1w1+x2w2+b
x_data= np.random.rand(50,2)
y_data = ((x_data[:,1] > 0.5)*
( x_data[:,0] > 0.5))
模型 資料](https://image.slidesharecdn.com/ntcdeeplearning-160406082943/85/TENSORFLOW-48-320.jpg)
![Tensorflow
x_ = tf.placeholder(tf.float32,[None,2])
y_ = tf.placeholder(tf.float32,[None,1])
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
y = tf.nn.sigmoid(tf.matmul(x_,w)+b)
cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1- y_) * tf.log(1 - y) )
optimizer = tf.train.GradientDescentOptimizer(0.1)
train = optimizer.minimize(cross_entropy)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
for step in xrange(500):
sess.run(train,feed_dict={x_:x_data,y_:y_data})
print sess.run(cross_entropy)
sess.close()
Computational
Graph
Session](https://image.slidesharecdn.com/ntcdeeplearning-160406082943/85/TENSORFLOW-49-320.jpg)
![Computation Graph
# placeholder
x_ = tf.placeholder(tf.float32,[None,2])
y_ = tf.placeholder(tf.float32,[None,1])
# variable
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
# operations
y = tf.nn.sigmoid(tf.matmul(x_,w)+b)
# error function
cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1-
y_) * tf.log(1 - y) )
# optimizer
optimizer = tf.train.GradientDescentOptimizer(0.1)
train = optimizer.minimize(cross_entropy)
# initalizer
init = tf.initialize_all_variables()](https://image.slidesharecdn.com/ntcdeeplearning-160406082943/85/TENSORFLOW-50-320.jpg)
![Placeholder
0.70828883
0.27190551
0.89042455
0.63832092
0.11332515
0.00849676
0.73278006
0.37781084
0.292448
0.09819899
0.9802261
0.94339143
0.36212146
0.54404682
……..
……..
0!
1!
0!
0!
0!
1!
0!
…!
x_ y_
x_ = tf.placeholder(tf.float32,[None,2])
y_ = tf.placeholder(tf.float32,[None,1])](https://image.slidesharecdn.com/ntcdeeplearning-160406082943/85/TENSORFLOW-51-320.jpg)
![Variable
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
0.42905441
-0.43841863
b
0!
w](https://image.slidesharecdn.com/ntcdeeplearning-160406082943/85/TENSORFLOW-52-320.jpg)
![Computation Graph
• Initializer
init = tf.initialize_all_variables()
w
b
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
0.42905441
-0.43841863
0!](https://image.slidesharecdn.com/ntcdeeplearning-160406082943/85/TENSORFLOW-58-320.jpg)
Uploaded byNTC.im(Notch Training Center)
TENSORFLOW深度學習講座講義(很硬的課程)
本讲座介绍了深度学习的基本概念、原理及其应用，包括图像识别、艺术创作和语义理解等。同时，介绍了TensorFlow的功能和使用方法，如自动微分和GPU支持。讲座涵盖了机器学习的不同类型以及逻辑回归和神经网络的训练过程。
More Related Content
![[系列活動] 手把手打開Python資料分析大門](https://cdn.slidesharecdn.com/ss_thumbnails/python-171213042421-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to TENSORFLOW深度學習講座講義(很硬的課程)
![[students AI workshop] Pytorch](https://cdn.slidesharecdn.com/ss_thumbnails/pytorch1-170911183243-thumbnail.jpg?width=640&height=640&fit=bounds)
TENSORFLOW深度學習講座講義(很硬的課程)
- 1.
- 2. 簡介 • 深度學習是什麼？ • 深度學習的原理 • Tensorflow是什麼？
- 3.
- 4. ⼈人腦 vs 電腦 8 < : 3x+ 2y + 5z = 7 5x + 1y + 8z = 9 9x + 4y + 3z = 14
- 5. ⼈人腦 vs 電腦 貨櫃船 機⾞車
- 6. ⼈人腦 vs 電腦 • ⼈人腦優勢： – 影像、聲⾳音 – 語⾔言 – ⾃自我意識（⾃自決⼒力） – … • 電腦優勢： – 數學運算 – 記憶（儲存）能⼒力 – …
- 7. 深度學習 • ⼀一種機器學習的⽅方法 • ⽤用電腦模擬⼈人腦神經系統構造 • 讓電腦學會⼈人腦可做的事
- 8.
- 9.
- 10.
- 11.
- 12.
- 13. 深度學習可以做的事 • 作畫 • 寫詩 • 開⾞車 • 下棋 • ……
- 14.
- 15.
- 16. ⾮非監督式學習 機器學習模型 Beijing is thecapital of China. As China's capital, Beijing is a large and vibrant city. Tokyo is the capital of Japan. As Japan’s capital, Tokyo is a large and vibrant city. ……. 資料 結果
- 17.
- 18. 機器學習 監督式學習 Supervised Learning ⾮非監督式學習 Unsupervised Learning 增強式學習 ReinforcementLearning 深度學習 Deep Learning
- 19.
- 20.
- 21. 符號慣例 訓練資料 全部：X , Y 單筆：x(i),y(i) 機器學習模型 h 模型參數 w 輸出值 h(X) 正確答案 Y 對答案 E(h(X),Y) 如果答錯了， 要修正模型 X Y
- 22.
- 23. 訓練資料 X Y -0.47241379 0 -0.35344828 0 -0.30148276 0 0.33448276 1 0.35344828 1 0.37241379 1 0.39137931 1 0.41034483 1 0.44931034 1 0.49827586 1 0.51724138 1 …. ….
- 24. 機器學習模型 Sigmoid function h(x)= 1 1 + e (w0+w1x) w0 + w1x < 0 h(x) ⇡ 0 w0 + w1x > 0 h(x) ⇡ 1
- 25. 修正模型 • Error function: Cross Entropy E(h(X), Y ) = 1 m ( mX i y(i) log(h(x(i) )) + (1 y(i) )log(1 h(x(i) ))) h(x(i) ) ⇡ 0 and y(i) = 0 ) E(h(X), Y ) ⇡ 0 h(x(i) ) ⇡ 1 and y(i) = 1 ) E(h(X), Y ) ⇡ 0 h(x(i) ) ⇡ 0 and y(i) = 1 ) E(h(X), Y ) ⇡ 1 h(x(i) ) ⇡ 1 and y(i) = 0 ) E(h(X), Y ) ⇡ 1
- 26. w1 w0 修正模型 • 梯度下降: w0 w0–⌘ @E(h(X),Y ) @w0 w1 w1–⌘ @E(h(X), Y ) @w1 ( @E(h(X), Y ) @w0 , @E(h(X), Y ) @w1 )
- 27.
- 28.
- 29. 模擬神經元 n W1 W2 x1 x2 b Wb y nin = w1x1+ w2x2 + wb nout = 1 1 + e nin nin nout y = 1 1 + e (w1x1+w2x2+wb)
- 30. nout = 1 nout= 0.5 nout = 0(0,0) x2 x1 模擬神經元 nin = w1x1 + w2x2 + wb nout = 1 1 + e nin nin = w1x1 + w2x2 + wb nout = 1 1 + e nin w1x1 + w2x2 + wb = 0 w1x1 + w2x2 + wb > 0 w1x1 + w2x2 + wb < 0 1 0
- 31. ⼆二元分類：AND Gate x1 x2 y 0 0 0 0 1 0 1 0 0 1 1 1 (0,0) (0,1) (1,1) (1,0) 0 1 n 20 20 b -30 y x1 x2 y = 1 1 + e (20x1+20x2 30) 20x1 + 20x2 30 = 0
- 32.
- 33. ⼆二元分類：XOR Gate n -20 20 b -10 y (0,0) (0,1) (1,1) (1,0) 0 1 (0,0) (0,1) (1,1) (1,0) 1 0 (0,0) (0,1) (1,1) (1,0) 0 0 1 n1 20 20 b -30 x1 x2 n2 20 20 b -10 x1 x2 x1 x2 n1 n2 y 0 0 0 0 0 0 1 0 1 1 1 0 0 1 1 1 1 1 1 0
- 34. 訓練類神經網路 • ⽤用隨機值初始化模型參數w • ForwardPropagation – ⽤用⺫⽬目前的模型參數計算出答案 • 計算錯誤量（⽤用Error Function） • Backward Propagation – ⽤用錯誤量來修正模型
- 35. 訓練類神經網路 訓練資料 機器學習模型 輸出值 正確答案 對答案 如果答錯了， 要修正模型 初始化 Forward Propagation Error Function Backward Propagation
- 36. 初始化 • 將所有的W隨機設成-N～N之間的數 • 每層之間W的值都不能相同 x y n11 n12 n21 n22 W12,y W12,x b W11,y W11,b W12,b b W11,x W21,11 W22,12 W21,12 W22,11 W21,b W22,b z1 z2
- 37.
- 38. Error Function J =(z1log(n21(out)) + (1 z1)log(1 n21(out))) (z2log(n22(out)) + (1 z2)log(1 n22(out))) n21 n22 z1 z2 nout ⇡ 0 and z = 0 ) J ⇡ 0 nout ⇡ 1 and z = 1 ) J ⇡ 0 nout ⇡ 0 and z = 1 ) J ⇡ 1 nout ⇡ 1 and z = 0 ) J ⇡ 1
- 39. w1 w0 Gradient Descent w21,11 w21,11⌘ @J @w21,11 w21,12 w21,12 ⌘ @J @w21,12 w21,b w21,b ⌘ @J @w21,b w22,11 w21,11 ⌘ @J @w22,11 w22,12 w21,12 ⌘ @J @w22,12 w22,b w21,b ⌘ @J @w22,b w11,x w11,x ⌘ @J @w11,x w11,y w11,y ⌘ @J @w11,y w11,b w11,b ⌘ @J @w11,b w12,x w12,x ⌘ @J @w12,x w12,y w12,y ⌘ @J @w12,y w12,b w12,b ⌘ @J @w12,b (– @J @w0 , – @J @w1 )
- 40. Backward Propagation 11(in) = @J @n11(in) = @J @n21(out) @n21(out) @n11(in) + @J @n22(out) @n22(out) @n11(in) =( 21(in)w21,11 + 22(in)w22,11) @n11(out) @n11(in) = @J @n21(out) @n21(out) @n21(in) @n21(in) @n11(out) @n11(out) @n11(in) + @J2 @n22(out) @n22(out) @n22(in) @n22(in) @n11(out) @n11(out) @n11(in) = ( @J @n21(out) @n21(out) @n21(in) @n21(in) @n11(out) + @J2 @n22(out) @n22(out) @n22(in) @n22(in) @n11(out) ) @n11(out) @n11(in)
- 41.
- 42.
- 43. Tensorflow • https://www.tensorflow.org/ • TensorFlow是 Google 開發的開源機器學習 ⼯工具。 • 透過使⽤用Computational Graph，來進⾏行數 值演算。 • ⽀支援程式語⾔言：python、C++
- 44. Tensorflow 機器學習Library (ex, scikit-learn) TensorFlow 從頭開始 彈性 技術門檻 把資料整理好後，剩 下的就直接呼叫API 自行定義 ComputationalGraph， 並交由TensorFlow計算。 自己推導微分公式， 自己寫整個流程 低 低 高 高
- 45. Tensorflow • 彈性 – 只要是可以⽤用Computational Graph來表達的運 算，都可以⽤用Tensorflow來解。 • ⾃自動微分 – ⾃自動計算Computational Graph微分後的結果。 • 平台相容性 – 同樣的程式碼可⽤用CPU執⾏行，亦可⽤用GPU執⾏行。
- 46.
- 47.
- 48. Example : BinaryClassification n w1 w2 1 b y x1 x2 x1 x2 y y = 1 1 + e x1w1+x2w2+b x_data= np.random.rand(50,2) y_data = ((x_data[:,1] > 0.5)* ( x_data[:,0] > 0.5)) 模型 資料
- 49. Tensorflow x_ = tf.placeholder(tf.float32,[None,2]) y_= tf.placeholder(tf.float32,[None,1]) w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0)) b = tf.Variable(tf.zeros([1,1])) y = tf.nn.sigmoid(tf.matmul(x_,w)+b) cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1- y_) * tf.log(1 - y) ) optimizer = tf.train.GradientDescentOptimizer(0.1) train = optimizer.minimize(cross_entropy) init = tf.initialize_all_variables() sess = tf.Session() sess.run(init) for step in xrange(500): sess.run(train,feed_dict={x_:x_data,y_:y_data}) print sess.run(cross_entropy) sess.close() Computational Graph Session
- 50. Computation Graph # placeholder x_= tf.placeholder(tf.float32,[None,2]) y_ = tf.placeholder(tf.float32,[None,1]) # variable w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0)) b = tf.Variable(tf.zeros([1,1])) # operations y = tf.nn.sigmoid(tf.matmul(x_,w)+b) # error function cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1- y_) * tf.log(1 - y) ) # optimizer optimizer = tf.train.GradientDescentOptimizer(0.1) train = optimizer.minimize(cross_entropy) # initalizer init = tf.initialize_all_variables()
- 51. Placeholder 0.70828883 0.27190551 0.89042455 0.63832092 0.11332515 0.00849676 0.73278006 0.37781084 0.292448 0.09819899 0.9802261 0.94339143 0.36212146 0.54404682 …….. …….. 0! 1! 0! 0! 0! 1! 0! …! x_ y_ x_ = tf.placeholder(tf.float32,[None,2]) y_ = tf.placeholder(tf.float32,[None,1])
- 52. Variable w = tf.Variable(tf.random_uniform([2,1],-1.0, 1.0)) b = tf.Variable(tf.zeros([1,1])) 0.42905441 -0.43841863 b 0! w
- 53. Matrix Multiplication y =tf.nn.sigmoid(tf.matmul(x_,w)+b) 0.42905441 -‐0.43841863 w x_ 0.70828883 0.27190551 0.89042455 0.63832092 0.11332515 0.00849676 …. …. 0.184686 0.1021888 0.04489752 …. tf.matmul(x_,w)+b b 0!
- 54. Sigmoid y = tf.nn.sigmoid(tf.matmul(x_,w)+b) 0.54604071 0.52552499 0.51122249 …. 0.184686 0.1021888 0.04489752 …. tf.nn.sigmoid
- 55. Error Function E(h(X), Y) = 1 m ( mX i y(i) log(h(x(i) )) + (1 y(i) )log(1 h(x(i) ))) cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1- y_) * tf.log(1 - y) ) 0.54604071 0.52552499 … 0! 1! …! y_ y 1.4331052 -tf.reduce_sum(y_*tf.log(y))
- 56. Optimizer optimizer = tf.train.GradientDescentOptimizer(0.1) train= optimizer.minimize(cross_entropy) w w ⌘ @E(h(X), Y ) @w b b ⌘ @E(h(X), Y ) @b
- 57.
- 58. Computation Graph • Initializer init= tf.initialize_all_variables() w b w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0)) b = tf.Variable(tf.zeros([1,1])) 0.42905441 -0.43841863 0!
- 59. Session # create session sess= tf.Session() # initialize variable sess.run(init) # gradient descent for step in xrange(500): sess.run(train, feed_dict={x_:x_data,y_:y_data}) # fetch variable print sess.run(cross_entropy, feed_dict={x_:x_data,y_:y_data}) # release resource sess.close()
- 60. Run Operations sess.run(init) the Nodein Computational Graph
- 61. Run Operations for stepin xrange(500): sess.run(train, feed_dict={x_:x_data,y_:y_data} ) the Node in Computational Graph Input Data 0.70828883 0.27190551 0.89042455 0.63832092 0.11332515 0.00849676 0.73278006 0.37781084 …….. …….. 0! 1! 0! 0! …! x_data y_data
- 62. Run Operations print sess.run(cross_entropy,feed_dict={x_:x_data,y_:y_data}) the Node in Computational Graph Input Data 0.70828883 0.27190551 0.89042455 0.63832092 0.11332515 0.00849676 0.73278006 0.37781084 …….. …….. 0! 1! 0! 0! …! x_data y_data Results 2.4564333
- 63. Training for step inxrange(500): sess.run(train, feed_dict={x_:x_data,y_:y_data} )
- 64. Demo : BinaryClassification https://github.com/ckmarkoh/ntc_deeplearning_tensorflow/ blob/master/intro/binaryClassification.ipynb
- 65. Tensorboard Histogram Summary Scalar SummaryComputational Graph
- 66. summary tf.scalar_summarytf.histogram_summary Histogram SummaryScalar Summary merged= tf.merge_all_summaries() writer = tf.train.SummaryWriter("./", sess.graph_def) for step in xrange(500): …. summary_str = sess.run(merged,feed_dict={x_:x_data,y_:y_data}) writer.add_summary(summary_str, step)
- 67. name_scope with tf.name_scope("cross_entropy") asscope: cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1-y_)*tf.log(1-y))
- 68. Launch Tensorboard > tensorboard--logdir=./ Starting TensorBoard on port 6006 (You can navigate to http://0.0.0.0:6006)
- 69.
- 70.
- 71. About the Speaker • Email: ckmarkoh at gmail dot com • Blog: http://cpmarkchang.logdown.com • Github: https://github.com/ckmarkoh Mark Chang • Facebook: https://www.facebook.com/ckmarkoh.chang • Slideshare: http://www.slideshare.net/ckmarkohchang • Linkedin: https://www.linkedin.com/pub/mark-chang/85/25b/847 71