A Neural Network is a computational model inspired by the structure and function of the human brain, designed to recognize patterns and solve complex problems. It deals with weight, bias, activation function, learning and loss function

1. Department of Artificial Intelligence and Data Science
Presented by
A.Kayalvizhi ( Sr.Gr ) / AI & DS
INTRODUCTION TO NEURAL NETWORKS AND PERCEPTRON
LEARNING ALGORITHM

2. What is a Neuron?
● An artificial neuron is also referred to as a perceptron. It is a mathematical
function which takes one or more inputs that are multiplied by values called
“weights” and added together to calculate weighted sum of inputs.
● This value is then passed to a non-linear function, known as an activation
function, to become the neuron’s output.

3. Structure of Biological Neuron

4. Biological Neuron to Artificial Neuron

5. History and Evolution of Neural Network
1943
Mc-Culloch and
pitts
Developed first AI
model
1958
Rosenblatt
Developed perceptron
1972
Paul Werbos
Developed
backpropagation to solve
XOR problem
1992
Max Pooling
Introduced 3D object
recognition
2012
CNN wins ImageNet
boosting deep learning
2020
Transformers, Advance AI
in NLP and Generative
model

6. PERCEPTRON
- Frank Rosenblatt
Accepts Binary Inputs
and Binary Outputs
Supports Logic gates
Linear Separability

7. Introduction
● The Perceptron is one of the simplest artificial neural network
architectures, introduced by Frank Rosenblatt in 1957.
● It is primarily used for binary classification.
● It supports supervised machine learning.

8. Structure of Perceptron

9. Basic Components of Perceptron
A Perceptron is composed of key components that work together to process information and
make predictions.
● Input Features: The perceptron takes multiple input features, each representing a
characteristic of the input data.
● Weights: Each input feature is assigned a weight that determines its influence on the
output. These weights are adjusted during training to find the optimal values.
● Summation Function: The perceptron calculates the weighted sum of its inputs,
combining them with their respective weights.

10. ● Activation Function: The weighted sum is passed through the step function, comparing it to
a threshold to produce a binary output (0 or 1).
● Output: The final output is determined by the activation function, often used for binary
classification tasks.
● Bias: The bias term helps the perceptron make adjustments independent of the input,
improving its flexibility in learning.
● Learning Algorithm: The perceptron adjusts its weights and bias using a learning algorithm,
such as the Perceptron Learning Rule, to minimize prediction errors.
Basic Components of Perceptron

11. Types of Perceptron
1. Single-Layer Perceptron : It is a type of perceptron is limited to learning linearly separable
patterns. It is effective for tasks where the data can be divided into distinct categories
through a straight line. While powerful in its simplicity, it struggles with more complex
problems where the relationship between inputs and outputs is non-linear. ( It has only input
and output layer)
2. Multi-Layer Perceptron: It possess enhanced processing capabilities as they consist of two or
more layers, adept at handling more complex patterns and relationships within the data.
(It has input layer, Hidden layers and output layer)

12. Single layer perceptron
"Single-layer perceptron can learn only
linearly separable patterns."
Input layer: Accepts input features.
Output layer: Produces a single output (0 or 1).
There is no hidden layer.
Applications
● Simple binary classification (e.g., AND, OR gates).
● Pattern recognition (when data is linearly separable).

13. Multilayer perceptron
A multi-layer perceptron model also has the same model structure but has a greater
number of hidden layers.
The multi-layer perceptron model is also known as the Backpropagation algorithm,
which executes in two stages as follows:
○ Forward Stage: Activation functions start from the input layer in the forward
stage and terminate on the output layer.
○ Backward Stage: In the backward stage, weight and bias values are modified as
per the model's requirement.

14. Multilayer perceptron

15. Linear separability - Perceptron
● The concept of linear separability involves separating or dividing the
input space into regions depending on whether the network output
response is positive or negative
● A decision line is drawn to separate the positive and negative responses.

16. Which one is linearly separable ?

17. Linear and Nonlinear separability

18. Convert the
summation
value to
zero
Convert it
to the form
of line
equation
Compute x-
intercept
with y = 0
Compute y-
intercept
with x = 0
Apply in the
2D plane
Draw the
decision
boundary
line
y = m x + c
Linear separability equation derivation

19. Linear Separability using Logic gates (AND and OR)
Both the AND and OR bitwise datasets are linearly separable, meaning that we can
draw a single line (green) that separates the two classes.

20. Non-Linear Separability using Logic gate (XOR)
However, for XOR it is impossible to draw a single line that separates the two
classes — this is therefore a nonlinearly separable dataset.

21. Perceptron Learning Algorithm
Perceptron learning model is a combination of two concepts
1. McCulloch-pitts model for an artificial neuron
2. Hebbian rule for adjusting weights

22. 1. Initialize weights (w), bias (b) and learning rate (η)
2. For each training sample (x,t):
Compute the weighted sum of inputs:
3. Apply the activation function (sign function):
The output y is:
θ = threshold (often taken as 0)
Perceptron Learning Algorithm

23. 4. Update weight and bias when y ≠ t:
5. Repeat steps 2-5 for all samples until all samples are classified correctly
Perceptron Learning Algorithm

24. Pictorial representation of Perceptron learning algorithm

25. Feed-Forward Neural Network

26. Process in Feedforward Neural Network
1. Initialization of Weights (W) and biases (b), Learning rate (η) and number of epochs are set.
2. Input features (X) are fed into the input layer.
3. Weighted Sum Calculation for each neuron
where:
● W = weight vector for that neuron
● X = input vector to that neuron
● b = bias for that neuron
1. The weighted sum (Yin) is passed through an activation function (f)
2. Perform Layer-by-Layer Forward Propagation.
3. Output Generation

27. 7. Error Calculation
● Compute the loss/error using a loss function:
○ Mean Squared Error (MSE) for regression:
8. Perform Backpropagation after forward pass
9. Repeat for All Epochs
Process in Feedforward Neural Network

28. Back propagation

29. Process in Backpropagation
1.Forward Pass
● Input x is passed through the network layer by layer.
● For each neuron:
● Apply activation function and until the final output is obtained.
2. Compute the Loss (Error)
Calculate the difference between predicted output and target output (y) using a loss
function
(e.g., Mean Squared Error, Cross-Entropy).

30. 3. Backward Pass (Error Propagation)
Calculate gradients of the loss with respect to each weight and bias using partial derivatives
(chain rule).
4. Update Weights and Biases
Using Gradient Descent:
For weights:
For biases:
5. Repeat
This process is repeated for multiple epochs over your dataset until the network's loss reduces.
Process in Backpropagation

31. Algorithm : Perceptron Learning Algorithm
P ← Inputs with label 1;
N ← Inputs with label 0;
Initialize w randomly;
while ! convergence do
Pick random x P U N
ϵ ;
if x P
ϵ and t then
w = w + x ;
end
if x P
ϵ and then
w = w + x ;
end
end
Note: The algorithm converges when all the inputs are classified correctly
Convergence Theorem