In HopebayTech.com Techfriday event, I share the brief understanding of Deep Learning concept which is learned from some great talks and reading.

1. Deep Learning
in a nutshell
JohnsonChen @ HB TechFriday

2. Agenda
1. Introduction to Deep Learning
2. Training You Own Deep Learning Model
a. Introduction to Tensorflow

3. What’s deep learning
1. Deep learning is part of a broader family of machine learning methods based
on learning representations of data.
2. Deep learning can be applied to binary classification problem ( ),
multiclass classification problem ( ).
3. Can we make machine learn by itself?
https://en.wikipedia.org/wiki/Deep_learning#Definitions

4. https://www.youtube.com/watch?v=4A14mOsT6vQ

5. Let’s talk about classification
http://www.louisdorard.com/blog/when-machine-learning-fails

6. Let’s talk about classification
( X1, X2, … )
Model

7. Build our own model
( X1, X2 )
(1, 1) +
(1, 0) +
(0, 1) +
(0, 0) -

8. (1, 1)
(1, 0)
(0, 1)
(0, 0)
This is our model

10. (1, 1)
(1, 0)
(0, 1)
(0, 0)
(0.3, 0.3)

11. What if ...
(1, 1) -
(1, 0) +
(0, 1) +
(0, 0) -

12. (1, 1)
(1, 0)
(0, 1)
(0, 0)

13. To make it “non-linear”

15. Activation Function
(relu: rectified linear unit)

16. “non-linear”

17. So what is deep learning?

18. Artificial Neural Network (ANN)
http://cs231n.github.io/neural-networks-1/

19. http://hyinfo.bse.ntu.edu.tw/labweb/ann.htm

20. Deep Learning (Deep Neural Network, DNN)
Deep = many hidden layers
6 ~ 8 hidden layers
http://markus.com/deep-learning-101/

21. https://classroom.udacity.com/courses/ud730/

22. https://classroom.udacity.com/courses/ud730/

23. ImageNet training set
http://www.slideshare.net/AmazonWebServices/cmp305-
deep-learning-on-aws-made-easycmp305
BIG DATA
BIG MODEL

24. How to train model?

25. (0, 1)
(0, 0)
(1, 1)
(1, 0)

26. Let’s try handwriting recognition
Most of slides are borrowed from Hung-yi Lee deep learning talk.

27. Handwriting recognition
https://www.tensorflow.org/versions/r0.8/tutorials/mnist/beginners/index.html#the-mnist-data

29. 784
28 * 28 = 784

30. Probability
Answer

35. Agenda
1. Introduction to Deep Learning
2. Training you own Deep Learning model
a. use Tensorflow
Most of slides are borrowed from Dr. Chung-Cheng Chiu deep learning talk

38. TensorFlow
Developed by Google Brain Team
Initial release: November 9, 2015
Used for both Google production and research.
Production: 50 different teams in dozens of commercial Google products, such as Google Voice, Gmail, Google
Photos, and Search, etc
Feature:
Python, C/C++ API
support multiple CPUs and GPUs
support mobile computing platforms, including Android and Apple's iOS

39. TensorFlow = Tensor + Flow
Tensor: n-dimensional arrays
Flow: A sequence of tensor operations
Deep Learning is suitable for TensorFlow, but TensorFlow can do more

40. Tensor

41. They are all tensor

42. Implement it
784 300 100 10
w1 w2 w3
b1 b2 b3
28 * 28 = 784
0
0
0
0
0
1
0
0
0
0

43. Implement it
Traning Data

44. Implement it
1. Define variables
2. Create (deep) neural network
3. Define cost function, optimizer
4. Train model parameter

45. Python code

46. Define variables
784 300 100 10
x y
w1 w2 w3
b1 b2 b3

47. Create (deep) neural network
784 300 100 10
x y
w1 w2 w3
b1 b2 b3

48. Define cost function, optimizer

49. Train model parameter

51. Save your model
# defaults to saving all variables - in this case w and b
saver = tf.train.Saver()

52. Code Demo
https://github.com/ChinHui-Chen/tensorflow-nn-example/blob/master/
multilayer_perceptron_example_novar.py

53. Visualization
http://scs.ryerson.ca/~aharley/vis/fc/

55. More examples

56. https://www.theinformation.com/How-Deep-Learning-Works-at-Apple-Beyond

57. AlphaGo: Deep Learning vs
AlphaGo Google DeepMind
3 9 3 15 AlphaGo
5
12
1. 3 9 ( )
2. 3 10 ( )
3. 3 12 ( )
4. 3 13 ( )
5. 3 15 ( )
Read: Google

58. Conclusion
Neural network, Deep learning
Create a framework to let machine learn by itself
Sometimes it is too complex to debug
TensorFlow tool
Try to train you own model.
Define input, out, and run/train it

59. http://www.techinsider.io/what-is-google-tensorflow-2015-11

60. Thanks