Feed forward neural networks use multiple perceptrons arranged in layers to perform increasingly complex functions. They take a weighted sum of inputs and pass them through an activation function to produce an output. The network learns through a backpropagation algorithm which calculates error rates to adjust weights between layers to minimize errors. Overfitting can occur if the network learns patterns specific to the training data and does not generalize well to new data.
3. An Artificial Neural Network (ANN) is a system that is based
on a biological neural network (brain).
▫ The brain has approximately 100 billion neurons, which
communicate through electrochemical signals
▫ Each neuron receives thousands of connections (signals)
▫ If the resulting sum of signals surpasses a certain threshold, the
response is sent
The ANN attempts to recreate the computational mirror of
the biological neural network …
ARTIFICIAL NEURAL NETWORK
4.
5. A perceptron models a neuron
It receives n inputs (feature vector)
It sums those inputs, calculated, then
output
Used for linear or binary classification
WHAT IS A PERCEPTRON?
6. The perceptron consists of weights, the summation processor,
and an activation function
A perceptron takes a weighted sum of inputs and outputs:
PERCEPTRON
7. Bias can also be treated as another input
▫ The bias allows to shift the line
The weights determine the slope
WEIGHTS AND BIASES
8. The transfer function translates the input signals into output signals
It uses a threshold to produce an output
Some examples are
▫ Unit Step (threshold)
▫ Sigmoid (logistic regression)
▫ Piecewise linear
▫ Gaussian
TRANSFER/ACTIVATION FUNCTIONS
9. The output is set depending on whether the total input is greater or
less than some threshold value.
UNIT STEP(THRESHOLD)
10. The output is proportional to the total weighted output.
PIECEWISE LINEAR
11. It is used when the output is expected to be a positive number
▫ It generates outputs between 0 and 1
SIGMOID
12. Gaussian functions are bell-shaped curves that are continuous
It is used in radial basis function ANN
▫ Output is real value
GAUSSIAN
13. To update the weights and bias to get smaller error
Help us control how much we change the weight and bias
THE LEARNING RATE
14. Initialize the weights (zero or small random value)
Pick a learning rate (0 – 1)
For each training set
Compute the activation output
▫ Adjusting
Error = differences between predicted and actual
Update Bias and Weight
Repeating till the error is very small or zero
If the output is linearly separable, we have found a solution
HOW THE ALGORITHM WORKS?
15. Because SLP is a linear classifier and if the data are not linearly
separable, the learning process will never find the solution
For example: the XOR problem
WHAT IF DATA IS NON-LINEARLY SEPARABLE?
16. A series of logistic regression models are stacked on top of each other.
The final layer is either another logistic regression or a linear regression
model, depending on whether we are solving a classification or
regression problem.
MULTILAYER PERCEPTRON (MLP)
17.
18. Use output error, to adjust the weights of inputs at the output layer
Calculate the error at the previous layer and use it to adjust the
weights
Repeat this process of back-propagating errors through any
number of layers
THE BACKPROPAGATION ALGORITHM
19. OVERFITTING PROBLEM
Overfitting occurs when you achieve a good fit of your model on the
training data, while it does not generalize well on new, unseen data.
In other words, the model learned patterns specific to the training
data, which are irrelevant in other data.
The best option to reduce overfitting is to get more training data.
Another way to reduce overfitting is to lower the capacity of the
model to memorize the training data.