Feed forward neural networks use multiple perceptrons arranged in layers to perform increasingly complex functions. They take a weighted sum of inputs and pass them through an activation function to produce an output. The network learns through a backpropagation algorithm which calculates error rates to adjust weights between layers to minimize errors. Overfitting can occur if the network learns patterns specific to the training data and does not generalize well to new data.

1. FEED FORWARD NEURAL NETWORK
1
ASHIKA DEVI R
NOORAIN FATHIMA
SHUPRIYA H
SRIDHAR GD

2.  Artificial Neural Network
 Perceptron Algorithm
 Multi-layer perceptron (MLP)
 Overfitting
CONTENTS

3.  An Artificial Neural Network (ANN) is a system that is based
on a biological neural network (brain).
▫ The brain has approximately 100 billion neurons, which
communicate through electrochemical signals
▫ Each neuron receives thousands of connections (signals)
▫ If the resulting sum of signals surpasses a certain threshold, the
response is sent
 The ANN attempts to recreate the computational mirror of
the biological neural network …
ARTIFICIAL NEURAL NETWORK

5.  A perceptron models a neuron
 It receives n inputs (feature vector)
 It sums those inputs, calculated, then
output
 Used for linear or binary classification
WHAT IS A PERCEPTRON?

6.  The perceptron consists of weights, the summation processor,
and an activation function
 A perceptron takes a weighted sum of inputs and outputs:
PERCEPTRON

7.  Bias can also be treated as another input
▫ The bias allows to shift the line
 The weights determine the slope
WEIGHTS AND BIASES

8.  The transfer function translates the input signals into output signals
 It uses a threshold to produce an output
 Some examples are
▫ Unit Step (threshold)
▫ Sigmoid (logistic regression)
▫ Piecewise linear
▫ Gaussian
TRANSFER/ACTIVATION FUNCTIONS

9.  The output is set depending on whether the total input is greater or
less than some threshold value.
UNIT STEP(THRESHOLD)

10.  The output is proportional to the total weighted output.
PIECEWISE LINEAR

11.  It is used when the output is expected to be a positive number
▫ It generates outputs between 0 and 1
SIGMOID

12.  Gaussian functions are bell-shaped curves that are continuous
 It is used in radial basis function ANN
▫ Output is real value
GAUSSIAN

13.  To update the weights and bias to get smaller error
 Help us control how much we change the weight and bias
THE LEARNING RATE

14.  Initialize the weights (zero or small random value)
 Pick a learning rate (0 – 1)
 For each training set
 Compute the activation output
▫ Adjusting
Error = differences between predicted and actual
Update Bias and Weight
 Repeating till the error is very small or zero
 If the output is linearly separable, we have found a solution
HOW THE ALGORITHM WORKS?

15.  Because SLP is a linear classifier and if the data are not linearly
separable, the learning process will never find the solution
 For example: the XOR problem
WHAT IF DATA IS NON-LINEARLY SEPARABLE?

16. A series of logistic regression models are stacked on top of each other.
The final layer is either another logistic regression or a linear regression
model, depending on whether we are solving a classification or
regression problem.
MULTILAYER PERCEPTRON (MLP)

18.  Use output error, to adjust the weights of inputs at the output layer
 Calculate the error at the previous layer and use it to adjust the
weights
 Repeat this process of back-propagating errors through any
number of layers
THE BACKPROPAGATION ALGORITHM

19. OVERFITTING PROBLEM
 Overfitting occurs when you achieve a good fit of your model on the
training data, while it does not generalize well on new, unseen data.
In other words, the model learned patterns specific to the training
data, which are irrelevant in other data.
 The best option to reduce overfitting is to get more training data.
 Another way to reduce overfitting is to lower the capacity of the
model to memorize the training data.

20. OVERFITTING PROBLEM

21. THANK YOU