The document provides an introduction to deep learning using TensorFlow, covering essential concepts like artificial intelligence, machine learning, and neural networks. It details the mechanics of neural networks, including activation functions, training through backpropagation, and architectures such as convolutional neural networks. Additionally, it discusses the advantages of TensorFlow as a machine learning library, highlighting its open-source nature and wide adoption in various industries.

1. Deep Learning using TensorFlow:
An Intro
Siby Jose Plathottam

2. Outline
AI and Machine LearningI.
Deep learning basicsII.
TensorFlowIII. basics
TensorFlowIV. demo

3. AI and ML: Artificial Intelligence
First coined by John McCarthy in 1956 at Dartmouth College.
What is AI?
Machines preforming task that are characteristic of human intelligence.
Machines learning and improving themselves.
Machine ’s response becoming indistinguishable from human response.
??
AGI vs ANI
Rule Based Heursitics Learning
AI Programs

4. AI and ML: Machine Learning (ML)
 First coined by Arthur Samuel in 1959 –
 “the ability to learn without being explicitly programmed”
 A concept? An algorithm? A mathematical formula?
 A computer programming paradigm:
Computer
Data
Output
Program

5. AI and ML: Comparison ML with conventional programs
Conventional programming approach
Machine learning approach
Programmer learns all the
steps in a specific task
Writer instructions for
each step
Machine executes
instructions
Programmer writes instructions for a
learning algorithm.
Machine figures out by itself (or doesn’t) how
to perform the assigned task.
Computer
Data
Program
Output

6. AI and ML: Top level view of Machine Learning
Supervised Reinforcement Unsupervised
Learning
Concepts
Tree based
Kernel
methods
Neural
Networks
Learning Models/
Algorithms
Linear
Nearest
Neighbour

7. AI and ML: Classification and Regression
ML tasks can be grouped into four types: classification, regression, clustering
and dimensionality reduction.
Classification:
Assign a discreet class/label to given input.
Examples: image and speech recognition, sentiment analysis, spam filtering etc.
Regression:
Predict continuous values using given input.
Examples: compute steering angle for autonomous vehicles, predict stock market
price etc.

8. Deep Learning basics: Artificial Neuron?
Originally known as the Perceptron and invented by Frank Rosenblatt in 1957.
Also referred to as a unit.
Biologically inspired computing unit.
Multiply all the inputs with parameters known as weights.
Sum all the weighted inputs and add to it a parameter known as the bias.
Pass the resulting scalar through a function known as the activation function.
A single neuron always produces a scalar output.

x1
x2
xn
W2
B
f(x)
y
x1W1 +x2W2 +….xnWn
+ B

9. Deep Learning basics: Feed Forward Neural Network
x1
x1
xn
Arrange multiple neurons in parallel to form a layer.
A single layer produces a vector as output.
Arrange multiple layers hierarchically to form an Feed Forward/Fully Connected ANN.
Simplest for of ANN: Feed Forward Neural Network
y1
y2
ym
Hidden layer
Output layer

10. Deep Learning basics: Weights, Biases, and
Matrix operations
Weights + biases: Store information
Inputs and weights can be represented as matrices.
Operations within a single layer: matrix multiplication + addition
Example for a layer with two inputs and three neurons.
 
1 1 11 2 21 1
11 11 11'
1 2 2 1 12 2 22 2
21 11 11
3 1 13 1 23 3
T T
B I W I W B
W W W
y I I B I W I W B
W W W
B I W I W B
    
                       

11. Deep Learning basics: Activation functions
Simple non -linear functions.
Properties:
Squishes the input
Active and passive regions.
Popular activation function:
Sigmoid
Tanh
Rectified linear unit
-5 -4 -3 -2 -1 0 1 2 3 4 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
Input
Output
Linear
Sigmoid
Tanh
ReLU

12. Deep Learning basics: Training and Inference
Two modes of ANN operation:
Training : adjusting weights and biases within ANN so that it produces useful
w.r.t. a given data set.
Optimization problem.
Iterative process which takes significant amount of time and computing resources.
Optimizer
Input Data
Output
Data
Untrained
Neural
Network
Trained
Neural
Network

13. Deep Learning basics: Training and Inference
 Inference: Use trained network to produce output for a data set it has
never seen before.
 Simple matrix operations.
 Takes only a very small fraction of the time taken by training.
Input Data
Trained
Neural
Network
Output
Data

14. Deep Learning basics: Backpropagation
Optimization algorithm to perform supervised training neural networks
Invented by David Ruelhart in 1986
Find error in ANN output
Use chain rule to find contribution of parameters to error
Update weights using gradient descent
Example: ANN with one input, 2 hidden neurons and one output neuron
 f(N1)
x1
x2
y1
N1=x1W11 +x2W22 + B1
B1
 f(N2)
y2
N2=x1W12 +x2W22 + B2
B2

O1=y1Wo1 +y2Wo2 + Bo
Bo
f(O1)
yo
yT
X1
X2

15. Deep Learning basics: Backpropagation (cont..)
Error is defined by:
Example: find contribution of weight Wo1 to error using chain rule,
Update weight:
Repeat for all weights for N iterations.
 
21
2
T oerror y y 
 
 
 
    
2
1
1 1 1
'
1 1
1
0.5
2 0.5 1
T o T o o
O T o o o
T o
O
y y y y y OE
W y y y O W
E
y y f O y
W
     

     

   

1
1 1
1
O O
O
E
W W
W


 


16. Deep Learning basics: Deep Feed Forward Networks
Deep Neural Networks (DNN): ANN ’s with more than one hidden layers
x1
x1
xn
y1
y2
ym
Hidden layer 1
Output layer
Hidden layer 2
Hidden layer n

17. Flattening
Feed Forward Layer
Convolutional Layer
SlidingWindow
Convolution
Filter
Feature Map
Input Layer Hidden Layer
1-DArray
2-D Array
Deep Learning basics: Convolutional
Neural Network (CNN)
Convolutional filter: a neuron
which which only operate on a
specific set of inputs at a time.
Receptive field: array size of the
weight matrix.
Feature map: output matrix
created by a Conv Filter.
Stride: size of step taken by the
filter when creating feature map

18. Deep Learning basics: Convolutional Neural Network (CNN)
x11 x12 x13 x14
x21 x22 x23 x24
F11F12
F21F22
F11F12
F22
F21 F22
F24 F25
F21 F22
F24 F25
F21
x31 x32 x33 x34
x41 x42 x43 x44
x11F11+x12F12
+
x21F21+x22F22
x12F11+x13F12
+
x22F21+x23F22
x13F11+x14F12
+
x23F21+x24F22
x21F11+x22F12
+
x31F21+x32F22
x22F11+x23F12
+
x32F21+x33F22
x23F11+x24F12
+
x33F21+x34F22
x31F11+x32F12
+
x41F21+x42F22
x32F11+x33F12
+
x42F21+x43F22
x33F11+x34F12
+
x43F21+x44F22
Feature Map 1
x11F11+x12F12
+
x21F21+x22F22
x12F11+x13F12
+
x22F21+x23F22
x13F11+x14F12
+
x23F21+x24F22
x21F11+x22F12
+
x31F21+x32F22
x22F11+x23F12
+
x32F21+x33F22
x23F11+x24F12
+
x33F21+x34F22
x31F11+x32F12
+
x41F21+x42F22
x32F11+x33F12
+
x42F21+x43F22
x33F11+x34F12
+
x43F21+x44F22
Feature Map n
Image data as a
matrix
OutputLayerwithSigmoidorSoftmaxActivation
L1LnL2
Class1
L1LnL2
ClassN
NLayerFeedForwardNetwork
ReLUActivation
ConcatenateFeaturemaps
Conv Filter 1
Conv Filter n

19. DATA
COMPUTING
ALGORITHMS
Deep Learning basics: Why now?
Amount of digital data available
has risen exponentially.
Computing resources: GPU ’s,
Cloud computing
Better algorithms/models:
Convnets, LSTMS, ReLU

20. TensorFlow basics: Machine Learning Libraries
 A menagerie of machine learning libraries.
 Developed in different programming languages
 Different front ends (API)
 Supported by firms, universities, individuals
https://developer.nvidia.com/deep-learning-frameworks

21. TensorFlow basics: Why TensorFlow?
Pros:
Google!
Opensource
Multi OS -> Max OS X, Linux, Windows, Android (compiled)
Multi Processor -> CPU, GPU, TPU, Cloud
Production ready code.
Exhaustive and evolving library (currently Version 1.2 is in pre-release)
Lot of firms (Intel, Airbus, Qualcomm …) and universities (Stanford, MIT,..) are using
it.
Cons:
Only about 18 months old.
Not possible to modify graphs during run time.
Programming methodology is not straight forward.

22. TensorFlow basics: TensorFlow Architecture
http://www.networkworld.com/article/3171272/software/google-opens-source-tensorflow-1-
0-debuts-vies-for-platform-status.html

23. TensorFlow basics: TensorFlow routine
Make Computational
Graph
Create a Session and
Execute Graph

24. Demo problem: Predict earthquake magnitude
using a Deep Feed Forward Neural
 Inputs: Latitude, longitude, depth
 Output: Magnitude in richter scale
 Code available in my GitHub:
https://github.com/sibyjackgrove/Earthquake_predict

25. Demo problem: Classify bird images using Convolutional
Neural Network Deep Feed Forward Neural
 Input: Images of two species of birds
 Output: Binary label
 Code available in my GitHub:
https://github.com/sibyjackgrove/CNN_for_bird_classification

26. References
S. S.1. Haykin and Simon, Neural networks: a comprehensive foundation, 2nd
ed. Prentice Hall, 1999.
Y.2. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no.
7553, pp. 436–444, May 2015.
A.3. Géron, Hands-on machine learning with Scikit-Learn and TensorFlow:
concepts, tools, and techniques to build intelligent systems, 1st ed. O’Reilly
Media, Inc, 2017.
J. Dean and R.4. Monga, “TensorFlow - Google’s latest machine learning
system, open sourced for everyone,” Google Research Blog, 2015. [Online].
Available: https://research.googleblog.com/2015/11/tensorflow-googles-
latest-machine_9.html.