This document provides an introduction to deep learning, including: 1. It discusses several applications of deep learning like image captioning, machine translation, virtual assistants, and self-driving cars. 2. It explains that deep learning can automatically extract complex patterns from large amounts of data using neural networks with many layers, unlike traditional machine learning which relies on human-designed features. 3. It outlines some of the challenges of training deep neural networks like overfitting and vanishing gradients, and how techniques like dropout, batch normalization, and improved optimization algorithms help address these issues.

1. INTRODUCTION TO DEEP LEARNING

2. Big Guns

4. Google Trends

5. Real ?

6. Look Around !

7. FR - Image Captioning API

8. Deep Learning is well descriptive

9. For Visually Impaired People

10. Self Driving Cars - Detection Mechanisms
MIT 6.S094- Self Driving

11. So Many Applications
1. How FB cluster images - Open Face
2. Google Neural Machine Translation - Seq2Seq
3. Google Assistant/ Siri - Seq2Seq , Attention Mechanisms
4. Self Driving Cars - Detection , Deep Reinforcement Learning
5. Signal Processing - Discriminative and separable feature extractors

13. Yes… Even In Wall Street !
Algorithmic Trading With Bots

14. THE GAP BETWEEN ACADEMIA AND
INDUSTRY IS REALLY REALLY LESS

15. Where To Start ?
● This is different from traditional Machine Learning
● SVM , Decision Trees , Random Forest , Naive Bayes , Regression

16. Where to use traditional ML ?
● Good for problems with linear classification boundaries

17. Not Lucky Enough ..
● Non Linear Boundaries
● We need algorithms with more non-linearity

18. Problems Getting Complex With Big Data
1. Data is new oil
2. Economy is data driven
3. So many patterns
4. So many connections

19. Mining the Big Data
Again more NON LINEARITY
Best Fit is Deep Neural Architecture

20. AI vs ML vs DL
● Nutshell

21. Machine Learning Basics - Very Brief
1. Supervised Learning - Neural Nets , SVM , Naive Bayes , Random Forest
2. Unsupervised Learning - K Means Clustering
3. Reinforcement Learning - Model Free Learning , MDP , Q Learning
4. Semi Supervised Learning - GAN(New)

22. Supervised vs Unsupervised

23. Reinforcement Learning

24. Semi Supervised Learning (GAN)

25. Again Why Deep Learning
❏ Traditional ML works with more rule based structure
❏ Not deep enough to extract complex patterns
❏ Should carefully input selected features : PCA , Handcraft -Haar
Haar Features
SIFT Features

26. Deep Learning Allows : End to End Training

27. Where to Start ..
★ Neural Architecture
Single Layer Neural Net

28. Architecture - Basics
● Classification Algorithm
● Trainable Weight Set
● Labeled Data

29. Let’s start with a node
Perceptron
Nonlinearity applied inside
a node

30. What Kind Of Nonlinearity ?

31. Nonlinearity to mine complex boundaries
Basic Idea -:
These units are there to squash the information which means they have a
working rage .
Ex : Activation Range Of Sigmoid (a) & Tanh(b) Neurons
Linear line in the curve

32. Understanding the Neural Model
● Training Phase -: Supervised Learning | Labeled data
● Inference / Testing Phase -: Checking the model for unseen data

33. Architecture
Forward Pass
•Calculating the loss

34. Backward Pass
•Backpropagation Algorithm
•Distributing Gradients

35. Optimizing
•Reducing the loss
•Updating the weight matrix

36. Updating weights - SGD
Loss FunctionUpdating weights

37. YOU CAN’T TRAIN
IF THERE ARE NO GRADIENTS

38. Deep Neural Networks

40. This went deeper!
How ?
With the help of two superheros!

42. Deep Neural Networks

43. What makes this special?

44. Hierarchical Feature Representation

45. Why this feature thing is so effective ?

46. Then what happened ?
Deep Neural Nets became harder and harder to train!
DEEP NETS!
Y U NO EASY?

47. Number Of Parameters?

48. These nets are huge!
Many
layers
nodes
parameters
BUT DON’T KNOW
HOW MANY HIDDEN
LAYERS / NODES TO
USE?
WHEN YOU WANT TO
BUILD A NEURAL NET

49. Overfitting

50. Algorithm Fail To Generalize Things
❏ When predicting , algorithm should be very logical , it should take decisions
based on the patterns .
❏ Otherwise It will fail eventually on Unseen Data
❏ When networks become bigger and bigger they came up with overfitting issue
more and more

51. Common Regularization wasn’t enough
● Adding some kind of penalty when tweaking
parameters

52. Common Regularization Methods
Not Enough !
Left - L2 Regularization | Right - L1 Regularization

53. Then Came up with new methods
1. Dropout
2. Modified SGD for optimization - Adam , RMSprop , Adagrad
3. Different architectures like CNN

54. Killing the training process
Neuron in a neural network
Sigmoid activation function

55. Batch Normalization Helped DL a LOT!

56. A Neural Net should have more Unsaturated Neurons .
● Backpropagation is basically applying chain rule
● In-order to make it happen we need to calculate local gradients in each node
connection
Network with ReLU
activation function

57. Feature Engineering

58. Canny edge detection filter

60. Can we design set of features for
machines ??
No way!
We may design some high level features!
But our machines deal with PIXELS!
(In other domains like NLP also)

61. What if we let the machine to
extract it’s own features!

62. Deep Learning is all about End-End Training
Automated Feature Extraction
Visualization of self learned filters of a CNN. Each layer different
features

63. Some Cool Applications
Self driving cars
Generative Chatbots

64. Detectors

65. VQA networks