neural networks.pdf

Neural
Networks
Ben Wycliff Mugalu
(ML Engineer, Marconi Lab)
The Basics

Ben Wycliff Mugalu - ben12wycliff@gmail.com
Neural Networks
A Neural Network (NN) is a machine learning approach inspired by the way
in which the human brain performs a particular learning task.
NNs mimic the human brain
● Knowledge about the task is acquired in form of examples
● Interneuron connection weights are used to store the acquired
information (from training examples)
● As you learn (see more examples) the weights are modified to develop
a deeper understanding of the task given the examples
2

Understanding Neural Networks (The Neuron)
3
Dendrites
Nucleus
Axon
Neurons on their own are pretty much
useless They work together to
perform complicated tasks.

How neurons work together
4
Dendrites
Nucleus
Axon
Receive signals from
a previous neuron
Carry signal/synapses to dendrites of
another neuron

Neuron representation in machines
5
X1
neuron
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output signal

6
X1
neuron
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output signal
Values are usually
standardized or normalized
Output value can be:
● Continuous (height)
● Binary (yes/no)
● Categorical (classes)

7
X1
neuron
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output signal
Single Observation,
Think of it as a
logistic regression.

8
X1
neuron
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output signal
W1
Wn
W2
Neural networks learn by assigning
weights to the signals.
Synapses are assigned
weights

9
X1
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output signal
W1
Wn
W2
Neural networks learn by assigning
weights to the signals.
Synapses are assigned
weights
?

10
X1
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output signal
W1
Wn
W2
What happens in the neuron: step 1
?
Weighted sum of input
signals/values.

11
X1
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
Output
value
W1
Wn
W2
What happens in the neuron: step 2
?
Activation function
The neuron then passes or does not pass the
signal depending on the activation function
Y

The Activation function
12
There are many types of activation functions. In this tutorial, we look at four
types of activation functions:
● Threshold function
● Sigmoid function
● Rectiﬁer
● Hyperbolic Tangent (tanh)

Sigmoid function
13
y

Hyperbolic Tangent function (tanh)
14
y

Threshold function
15
y
1
0

Rectifier function
16
y
1
0

How neural networks work
17
Assuming we have a dataset that provides a set of attributes about various
houses in the neighborhood that include:
❖ Number of bedrooms
❖ Age
❖ Area
❖ Distance from main road
❖ Price
Our task is to develop a neural network that can predict the price of the
house using the number of bedrooms, age, area, and the distance from
main road

18
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Input layer

19
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Input layer
Y
Output layer
W1
W2
W3
W4

20
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Input layer
Y
Output layer
W1
W2
W3
W4
This is roughly a representation of a
traditional machine learning algorithm

21
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Input layer Hidden layer
The power of artiﬁcial neural networks is
in hidden layers.
Y
Output layer

22
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Neurons in the hidden network are used
to capture various features. Some of these
features might not make sense to a
human being.
Y
Output layer

23
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Rectiﬁer function in action
Y
Output layer
The last node captures the impact of the
number of bedrooms and the area of the
land on which the house is sitted. The
higher the weighted sum of X1 and X3, the
higher the output of our activation
function

How Neural Networks learn.
24
X1
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
W1
Wn
W2 Ỹ
Y
Predicted value
Actual value

25
X1
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
W1
Wn
W2 Ỹ
Y
Predicted value
Actual value
The output value is compared with the
actual value. Loss is acquired using a
cost function
N

26
X1
X2
Xn
⋮
Input value 1
Input value 2
Input value 3
W1
Wn
W2 Ỹ
Y
Predicted value
Actual value
The error is returned and utilized to
modify the weights.
N

Back propagation
27
● Backpropagation is the method of ﬁne-tuning the weights of a neural
network based on the error rate obtained in the previous iteration.
● It helps calculate the gradient of a loss function with respect to all the
weights in the neural network. ( We are able to adjust all the weights in
the network at the sametime).

How to minimize the loss (Brute force)
28
Cost
Ẁ
Plot the loss for all possible weights,
take the weights at minimum loss.
N

Disadvantage of Brute force
29
Consider neural network below
X1
X2
X3
X4
Bedrooms
Age
Area
Distance
Y
Output layer
Price
16 weights

Disadvantage of Brute force
30
● With 16 weights.
● Assuming we want to evaluate 1000 combinations of weights while
testing for every weight.
● We would have to evaluate a total of combinations.
● This would take approximately:
● On an i5, 8th generation core processor with 5.73 giga ﬂops

Gradient Descent
31
Cost
Ẁ
N

Gradient Descent
32
Cost
Ẁ
N

Gradient Descent
33
Cost
Ẁ
N

Gradient Descent
34
Cost
Ẁ
N

Gradient Descent
35
Cost
Ẁ
N

Gradient Descent
36
Cost
Ẁ
N

Gradient Descent
37
Cost
Ẁ
N

Gradient Descent
38
Cost
Ẁ
Global minimum
N

Types of Gradient Descent
● Batch Gradient Descent
● Stochastic Gradient Descent
● Mini-Batch Gradient Descent
39

Types of Neural Networks
Feed Forward Neural Networks
40
Single-layer Perceptron Multi-layer Perceptron

Convolution Neural Networks
41

Recurrent Neural Networks
42

Practical Example
https://gist.github.com/ben-wycliﬀ/07885b717ebe7f2818425d50e9f276fb
43
Single Input, Single Perceptron (Linear Regression)

Ben Wycliﬀ Mugalu
ML Engineer, Marconi Lab
ben12wycliff@gmail.com
The End

neural networks.pdf

More Related Content

Similar to neural networks.pdf

Recently uploaded

neural networks.pdf