1. Design
How to solve a problem using
Deep Learning
AI for Good Workshop
July 2019
Session 3
https://sites.google.com/view/AIforEveryone
2. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Key Learning objective
for Day 1
How to solve a problem with Deep Learning ?
How to design an neural network architecture?
3. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz #1
Is it possible to map hand signs for deaf speech
Hand signs Speech
4. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
A B
•
5. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz #2
Is it possible to “see the depth” for visual impaired
Yes / no ?
6. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz #2
Is it possible to feel the depth for visual impaired
7. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
8. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Topics in this presentation
•New way to think from today !
• How to use Deep Learning to solve any problem
• How to model & architect Neural Networks
9. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
New way to think from today !
10. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
New way to think from today !
11. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Deep Learning idea-a-thon:
How to solve a problem
by designing novel neural network architectures?
CNN
RNN
Fully
Connected
Merge
CNN is suitable architecture to extract
features out of any data that has
spatial relationships. For example, An
photo image spatial info..or info
spread across space
In general, a recurrent neural network could
be considered as the best neural network
model for extracting features out of
temporal data. A temporal data is basically
a data that varies over time such as ECG,
music, speech, sentences, words. LSTM is
a advanced RNN
To classify / predict
To combine the learnings of
two neural networks.
(to combine the power of 2
people’s brains)
Attention
To learn to focus on the
most important aspects to
achieve a particular task
Generative
Deep Learning
To model creative tasks as
to compose a music or
to come up a painting
12. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Our inspiration / acknowledgements
Friendly approaches :
1) KERAS.io
François Chollet’s
Book on “Deep Learning with Python”
2) Deeplearning.ai (Coursera.org)
Andrew Ng
3) Udacity Deep Learning Free course
4) Google Machine Learning Course
https://developers.google.com/machine-learning/crash-course/
13. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
More inspiration/ acknowledgements
Excellent Resources
• Stanford cs231 n
http://cs231n.stanford.edu
• MIT Deep Learning
http://introtodeeplearning.com/
https://deeplearning.mit.edu
• IIT Madras
my classes notes with Prof. Anurag (Deep Learning)
14. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Message: Using
Representation learning to
learn to represent A as B
15. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Learning objective:
How did the magic happen?
Deep
Learning
16. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Learning objective:
How did the magic happen?
17. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
18. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Learn to map Xy
• Examples of x, y
19. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
A B
20. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
21. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
22. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
23. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Fun activity
24. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Learning objective:
How do solve a problem
using Deep Learning?
Case study: self driving car
25. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Camera view ? steering angle
•
Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
Credits: Nvidia
26. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
•
27. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Learn to map x->y
28. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
What is the function to map x->y
•
29. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Key Learning objective
for Day 1
How to solve a problem with Deep Learning ?
How to design an neural network architecture?
30. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #1
classification problems
CNN
Fully
Connected
Prediction of what
of objects is in this
photo
31. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #1
CNN
Fully
Connected
Turn car left
Turn car right
Don’t Turn the car
32. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #1
33. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #1
34. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #2
Regression problems
CNN
Fully
Connected
Continuous number
35. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #2
Regression problems
CNN
Fully
Connected
Turn the car by 10 degrees
36. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #2
Regression problems
37. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #2
Regression problems
•
38. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #3
multimodal classification
CNN
Fully
Connected
Predicted
class
RNN
Merge
39. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #3
multimodal classification
40. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #3
multimodal classification
41. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
Neural Network Design Pattern #3
multimodal classification
•
Quiz: What was x and y in the problem?
42. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
Neural Network Design Pattern #3
• Quiz: What was x and y in the problem?
43. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
multimodal classification
44. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Answering Visual Questions from Blind People
45. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #3
multimodal classification
46. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #3
multimodal classification
•
47. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #3
48. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #4
video question answering
CNN
Fully
Connected
Predicted
class
RNN
Merge
Video
Question
49. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #4
video question answering
50. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #4
video question answering
•
51. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #4
video question answering
52. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #4
video question answering
53. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #4
video question answering
54. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #4
video question answering
55. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #4
video question answering
56. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #5
image question answering
CNN
Answer
(text)
RNN
Merge
Image
(photo)
Question
(text)
RNN
57. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #5
image question answering
58. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #6
59. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #6
60. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Architecture patterns
Neural Network Design Pattern #7
image to image translation
CNN
Image
(text)
Image
(photo)
CNN
Features of
the image
61. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #7
image to image translation
62. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #7
image to image translation
63. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #7
image to image translation
64. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Neural Network Design Pattern #7
image to image translation
65. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
Can a neural network output 2 types
of output?
• How to architect Multi output network?
66. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
Multi output network
67. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
how to extract features from multiple
sources ?
• I have a input of audio and video and text. How to combine the features
from all 3?
68. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Quiz
how to extract features from multiple sources ?
69. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
What did you learn?
(Day 1)
How to solve a problem with Deep Learning ?
How to design an neural network architecture?
70. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Key ideas to remember
You can map any thing to anything with Deep Learning AB
• Anything Machine Learns to map Anything
71. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Key ideas to remember
How to design a architectures to handle various types of A B
How to design this architecture?
CNN / RNN ?
Fully Connected or RNN ?
Predicted
class
RNN /
Fully connected ?
Merge / Attention ?
Video
Question
A -------------------------------------------------------------------------------------------------------------------------- B
72. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Key ideas to remember
From architecture to code ? (just learn early glimpse of how code looks like.)
Just 1 page of code is enough for a complex problems such as video question answering
73. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Key ideas to remember
If you didn’t understand the code*, just don’t worry.
• Why not to worry?
• Solving the video question answering problem was not possible until 2017.
• To solve this problem, it required dozens of experts in Google lab to dedicate 6 months of effort!
*code = code in the previous slide
74. Acknowledgments & Credits are mentioned to inspirational resources presented in the end of presentation. For more like this, https://sites.google.com/view/AIforEveryone
Team activity
idea-a-ton
Goal of this thinking activity
How to solve a problem with Deep Learning ?
How to design an neural network architecture?