The document discusses deep learning and its applications. It introduces neural networks and how they are used for tasks like image recognition. Deep learning techniques like convolutional neural networks can be used for computer vision in robotics. The document provides an overview of deep learning concepts and tools for implementing deep learning models, and provides an example of building a convolutional network model with TensorFlow.

1. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Deep Learning & Applications in Visual Sensing for
Robotics
Son N. Tran
June 21, 2016
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

2. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Plan
1 Deep Learning: A Brief Introduction
2 Machine Learning & Deep Learning for Beginers
3 Visual Sensing: How to Recognise Objects
4 Applications in Robotics
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

3. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Deep Learning Rumours
Deep Learning!!! Sound interesting, huh? Everyone is talking
about it.
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

4. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
DL in Games
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

5. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
DL in Weather Forecasting
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

6. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
DL in Healthcare
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

7. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
DL in Beauty Industry
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

8. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
DL in Robotics
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

9. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Neural Networks
So, what is deep learning???
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

10. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Neural Networks
Let’s start with Neural Networks.
Example:
x1 = 1, x2 = 0, x3 = 0, x4 = 1
⇒ h1 = f (2 ∗ 1 + 3 ∗ 0 + 2 ∗ 0 + 1 ∗ 1 + 0.5) = f (3.5).
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

11. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Neural Networks
Let’s start with Neural Networks.
linear: f (x) = x
logsig: f (x) = 1/(1 + e−x )
tansig: f (x) = ex −e−x
ex +e−x
relu (rectifier linear): f (x) = max(0, x + N(0, x)) where N is
Normal distribution (Gauss).
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

12. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Neural Networks
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

13. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Train Neural Networks
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

14. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Deeper Networks
More number of layers → better prediction?
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

15. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Deeper Networks
More number of layers → better prediction?
No
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

16. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Deeper Networks
More number of layers → better prediction?
No
Gradient exploding/diminishing
Overfitting
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

17. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Unsupervised Learning
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

18. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Unsupervised Learning
Autoencoder
Restricted Boltzmann
Machines
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

19. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Unsupervised Learning
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

20. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Stacking & Layer-wise Pretraining
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

21. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Rectifier Linear Units
relu (rectifier linear): f (x) = max(0, x + N(0, x)) where N is
Normal distribution (Gauss).
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

22. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Dropout
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

23. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Convolutional Neural Networks: Image input
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

24. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Convolutional Neural Networks: What is convolution?
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

25. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Convolutional Neural Networks: Stacking Component.
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

26. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Convolutional Neural Networks: A Model.
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

27. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Plan
1 Deep Learning: A Brief Introduction
2 Machine Learning & Deep Learning for Beginers
3 Visual Sensing: How to Recognise Objects
4 Applications in Robotics
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

28. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Skills
Programming skills (Python, C/C++, Lua, MATLAB,Java)
Matrix Calculation
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

29. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Tools
TensorFlow:
Convnet
Theano
Caffe
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

30. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Online courses
Coursera
Machine Learning
Neural Networks for Machine Learning
Udacity
Intro to Machine Learning
Machine Learning
Deep Learning
Youtube: mathematicalmonk - Machine Learning
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

31. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Private Courses & Mentoring
Email: piedu.team@gmail.com
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

32. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
TensorFlow
Tensor (multi-dimensional matrix) computation library.
Optimization algorithms
Reduce coding effort
Easy to deploy
Cross-platform support
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

33. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
Building graph:
1. with tf.Graph().as default():
2. images, labels = distorted inputs()
3. logits = build model(images)
4. loss = loss func(logits,labels)
5. train op = train batch(loss,global step)
6. for step in xrange(FLAGS.max steps):
7 , loss value = sess.run([train op, loss])
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

34. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
3. logits = build model(images)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

35. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
1. filters = tf.get variable(’weights’, shape=[5, 5,
3, 64],
initializer=tf.truncated normal initializer(stddev=1e-4))
2. conv op = tf.nn.conv2d(images, filters, [1, 1, 1,
1], padding=’SAME’)
3. biases = tf.get variable(’biases’, [64],
initializer=tf.constant initializer(0.0))
4. conv bias = tf.nn.bias add(conv op, biases)
5. conv1 = tf.nn.relu(conv bias, name=scope.name)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

36. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
1. filters = tf.get variable(’weights’, shape=[5, 5,
3, 64],
initializer=tf.truncated normal initializer(stddev=1e-4))
2. conv op = tf.nn.conv2d(images, filters, [1, 1, 1,
1], padding=’SAME’)
3. biases = tf.get variable(’biases’, [64],
initializer=tf.constant initializer(0.0))
4. conv bias = tf.nn.bias add(conv op, biases)
5. conv1 = tf.nn.relu(conv bias, name=scope.name)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

37. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
1. filters = tf.get variable(’weights’, shape=[5, 5,
3, 64],
initializer=tf.truncated normal initializer(stddev=1e-4))
2. conv op = tf.nn.conv2d(images, filters, [1, 1, 1,
1], padding=’SAME’)
3. biases = tf.get variable(’biases’, [64],
initializer=tf.constant initializer(0.0))
4. conv bias = tf.nn.bias add(conv op, biases)
5. conv1 = tf.nn.relu(conv bias, name=scope.name)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

38. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
1. filters = tf.get variable(’weights’, shape=[5, 5,
3, 64],
initializer=tf.truncated normal initializer(stddev=1e-4))
2. conv op = tf.nn.conv2d(images, filters, [1, 1, 1,
1], padding=’SAME’)
3. biases = tf.get variable(’biases’, [64],
initializer=tf.constant initializer(0.0))
4. conv bias = tf.nn.bias add(conv op, biases)
5. conv1 = tf.nn.relu(conv bias, name=scope.name)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

39. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
1. filters = tf.get variable(’weights’, shape=[5, 5,
3, 64],
initializer=tf.truncated normal initializer(stddev=1e-4))
2. conv op = tf.nn.conv2d(images, filters, [1, 1, 1,
1], padding=’SAME’)
3. biases = tf.get variable(’biases’, [64],
initializer=tf.constant initializer(0.0))
4. conv bias = tf.nn.bias add(conv op, biases)
5. conv1 = tf.nn.relu(conv bias, name=scope.name)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

40. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
1. pool1 = tf.nn.max pool(conv1, ksize=[1, 3, 3, 1],
strides=[1, 2, 2, 1], padding=’SAME’, name=’pool1’)
2. norm1 = tf.nn.lrn(pool1, 4, bias=1.0, alpha=0.001
/ 9.0, beta=0.75, name=’norm1’)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

41. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Coding Example: Convolutional Network with TensorFlow
3. logits = build model(images)
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

42. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Plan
1 Deep Learning: A Brief Introduction
2 Machine Learning & Deep Learning for Beginers
3 Visual Sensing: How to Recognise Objects
4 Applications in Robotics
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

43. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Object Localisation
Selective search: Demo
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

44. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Object Localisation
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

45. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Object Localisation
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

46. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Recognition model
Use Object Localisation techniques to generate candidate
objects.
Extract the bounding-boxes and use them to train another
Deep CNN for prediction
or
Train a single deep CNN for both localisation and recognition.
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

47. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Regconition: demo
Demo: http://demo.bagiks.com
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

48. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Bagiks Team
Bagiks team: http://www.bagiks.com/
Interested in using/helping to improve the solutions? contact
team@bagiks.com
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

49. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Plan
1 Deep Learning: A Brief Introduction
2 Machine Learning & Deep Learning for Beginers
3 Visual Sensing: How to Recognise Objects
4 Applications in Robotics
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

50. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Camera Sensor
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

51. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Driverless Car
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

52. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
Serving Robots
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

53. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
and more ...
Cleaner robot.
Personal assistant
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics

54. Deep Learning: A Brief Introduction
Machine Learning & Deep Learning for Beginers
Visual Sensing: How to Recognise Objects
Applications in Robotics
THANKS!
PiEdu
Bagiks
Thank you.
Contact Presenter: sontn.fz@gmail.com
Son N. Tran Deep Learning & Applications in Visual Sensing for Robotics