This document discusses machine learning classification using a single layer feed forward neural network. It begins with definitions of machine learning and the different types of machine learning problems. Supervised learning classification is explained where the goal is to learn from labeled training data to classify new observations. Common classification algorithms and the components of a learning model are described. Finally, the document provides an example of how a single layer perceptron neural network can be used to classify a sample dataset into two classes by learning the optimal weights through an iterative process.

1. Machine Learning using
Classification
Feed Forward Neural Network
1

2. Agenda
1. Machine Learning Definition.
2. Types of machine learning.
3. Classification Definition.
4. Classification Algorithms.
5. Components of learning.
6. Neural Network.
7. Single layer perceptron.
Feed Forward Neural Network
2

3. What is machine learning?
Definition of Machine Learning from WhatIS.com
“is a type of artificial intelligence (AI) that allows software
applications to become more accurate in predicting outcomes
without being explicitly programmed”.
Definition of Machine Learning from dictionary.com
“The ability of a machine to improve its performance based on previous
results.”
Feed Forward Neural Network
3

4. Types of machine Learning
Machine
learning
categorization
Supervised
Learning
UnSupervised
Learning
Enforcement
Learning
Regression /
classification
clustering
Feed Forward Neural Network
4

5. Supervised Learning “Classification”
In machine learning and statistics, classification is a supervised
learning approach in which the computer program learns from the
data input given to it and then uses this learning to classify new
observation.
?
Feed Forward Neural Network
5

6. Classification algorithms:
1. Logistic Regression.
2. Naïve Bayes.
3. K-Nearest Neighbors.
4. Decision Tree.
5. Support Vector Machine.
6. Neural Network.
Feed Forward Neural Network
6

7. Components of learning
• Example:-Suppose that you are a bank manger and you want to
predict your customers behavior in the future according to their
historical data in order to make a credit card for money withdrawing
or not (approve it or deny).
23 years
Age
Male
Gender
$30,000
Annual Salary
1 Year
Years in residence
1 Year
Years in job
$ 15,000
Current debt
Sample of data vector
Feed Forward Neural Network
7

8. Components of learning cont.
• Formalization:-
• Input : x (vector of data ‘Customer Application’).
• Output : y (good/bad customer).
• Target Function : 𝒇: 𝒙 → 𝒚 (ideal credit approval formula).
“it is unknown function as if it known we don’t need to learn the machine and just go
ahead and implement our algorithm”.
• Data : 𝒙𝟏, 𝒚𝟏 , 𝒙𝟐, 𝒚𝟐 , … 𝒙𝒏, 𝒚𝒏 (Historical data)
• Hypothesis: g: 𝒙 → 𝒚
• we will use the historical data above in order to get the hypothesis which is the formal
name we are going to call the formula we get.
Feed Forward Neural Network
8

9. Components of learning cont.
ideal credit approval formula
Historical records of credit customers
Learning
Algorithm
Unknown target function
𝒇: 𝒙 → 𝒚
Training Example
𝒙𝟏, 𝒚𝟏 , 𝒙𝟐, 𝒚𝟐 , … 𝒙𝒏, 𝒚𝒏
Final Hypothesis
𝒈 ≈ 𝒇
Hypothesis Set
𝑯
Set of candidate formula
Final credit Approval formula
Both of them are
called “Learning
Model”
Feed Forward Neural Network
9

10. • For input 𝑿 = (𝒙𝟏, 𝒙𝟐, 𝒙𝟑 , … , 𝒙𝒏) ‘Attributes of the customer’
Approve credit if 𝒊=𝟏
𝒏
wixi > 𝒕𝒉𝒓𝒆𝒔𝒐𝒍𝒅
Deny credit if 𝒊=𝟏
𝒏
wixi < 𝒕𝒉𝒓𝒆𝒔𝒐𝒍𝒅
So ℎ 𝑥 = 𝑠𝑖𝑔𝑛(( 𝒊=𝟏
𝒏
wixi )- threshold))
A simple hypothesis “The perceptron”
Feed Forward Neural Network
10

11. A simple hypothesis “The perceptron”
cont.
• ℎ 𝑥 = 𝑠𝑖𝑔𝑛(( 𝒊=𝟏
𝒏
wixi )- threshold))
• ℎ 𝑥 = 𝑠𝑖𝑔𝑛(( 𝒊=𝟏
𝒏
wixi ) + w0 ))
• ℎ 𝑥 = 𝑠𝑖𝑔𝑛( 𝒊=𝟎
𝒏
wixi )
Why W0 in order to get the following formula
by introducing an artificial coordinate
or vector called the bias x0=1
Feed Forward Neural Network
11

12. Neural Network “single layer perceptron”
• is a system that is based on the biological neural
network, such as the brain. The brain has
approximately 100 billion neurons, which
communicate through electro-chemical signals.
• Each neuron receives thousands of connections
with other neurons, constantly receiving
incoming signals to reach the cell body.
• If the resulting sum of the signals surpasses a
certain threshold, a response is sent through the
axon.
Feed Forward Neural Network
12

13. Neural Network “single layer
perceptron”cont.
• is comprised of a network of artificial neurons (also known as
"nodes").
• These nodes are connected to each other, and the strength of their
connections to one another is assigned a value based on their
strength: inhibition (maximum being -1.0 week connection) or
excitation (maximum being +1.0 strong connection) we call them
weights.
• Three types of nodes:
• Input nodes.
• Hidden nodes.
• Output nodes.
Feed Forward Neural Network
13

14. Neural Network “single layer
perceptron”cont.
• There are two types of neural network:
1. Single layer neural network “Perceptron”.
2. Multilayer neural network.
Feed Forward Neural Network
14

15. Neural Network “single layer
perceptron”cont.
Algorithm steps:
1. Start with the first sample.(Note: initial weights = 0)
2. Calculate Net = 𝒊=𝟎
𝒏
wixi .
EX:- Net = w1x1 + w2x2 + w0x0
Where x0 is a feature vector used to reduce number of iterations to get the optimal
line that used to separate the data it is called “Bias” its initial value =1.
3. Calculate f(Net) : which is called :
1. Transfer function. 2. Activation function.
Ex :- F(Net) =
1 𝑁𝑒𝑡 ≥ 0
0 𝑁𝑒𝑡 < 0
4. If Output = Desired then go to next sample.
else Output ≠ Desired Update weights then go to next sample.
5. Terminate step: find weights correct to all samples:
∆wi= η * ( Desired – Output) * xi
∆w = New weight - Old weight
New Weight = ∆w + Old Weight
η is called learning rate
“Eta” used also to reduce
number of iterations
especially during weights
updating initially = 1
Feed Forward Neural Network
15

16. Example :Single Layer Perceptron
• Consider the following training data set:
1. Is the problem solvably ?Why?!
2. If it is solvable find weights using
perceptron single layer? Given :
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0
1. Classify the following samples:
1. (-2.0) .
2. (1,1).
3. (0,1).
4. (-1,-2).
Desired
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
η=1
Feed Forward Neural Network
16

17. Example :Single Layer Perceptron cont.
Solution
1. The problem can be solved because it has only two classes (1, -1) ,so it can be solved
using single layer perceptron.
2. The problem can be solved using tow ways:
1. Using Graph :
Represent the data set points on x and y
and find separable line between those two classes.
𝑦−y1
𝑥−x1
=
y2−y1
x2−x1

𝑦− 1
𝑥+0.5
=
−1−1
−0.5+0.5
𝑦− 1
𝑥+0.5
=
−2
0
 -2x-1 = 0y  -2 x - 0y - 1 =0
w1x1 + w2x2 + w0x0
so w1 =-2 , w2 =0 , w0 =-1
-1
-2
1
2
-1
-2
1 2
(-0.5,1)
(-0.5,-1)
(-0.5,1) (-0.5,-1)
(x2, y2)
(x1, y1)
Feed Forward Neural Network
17

18. Example :Single Layer Perceptron cont.
2. Using perceptron :
1. Take the first sample, and initialize all weights to zero:
2. Calculate Net = 𝒊=𝟎
𝒏
wixi = 0*1 + 0*-1 + 0*1 = 0
f(Net) = -1
3. while f(net) ≠ desired so update weights and go to next sample
4. ∆w0= η * ( Desired – Output) * xi = 1*(1+1)*1 =2
w0new = ∆w0+ wold = 2 + 0 = 2
∆w1= η * ( Desired – Output) * xi = 1*(1+1)*-1 =-2
w1new = ∆w1+ wold = -2 + 0 = -2
∆w2= η * ( Desired – Output) * xi = 1*(1+1)*1 =2
w2new = ∆w2+ wold = 2 + 0 = 2  go to next sample
W2 = 0
W1 = 0
W0 = 0
x2 = 1
x1 = -1
x0 = 1
y
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
Feed Forward Neural Network
18
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

19. Example :Single Layer Perceptron cont.
1. Take the Second sample, and initialize all weights to zero:
2. Calculate Net = 𝒊=𝟎
𝒏
wixi = 2*1 + -2*0 + 2*1 = 4
f(Net) = 1
3. while f(net) ≠ desired so update weights and go to next sample
4. ∆w0= η * ( Desired – Output) * xi = 1*(-1-1)*1 =-2
w0new = ∆w0+ wold = -2 + 2 = 0
∆w1= η * ( Desired – Output) * xi = 1*(-1-1)*0 =0
w1new = ∆w1+ wold = 0 -2 = -2
∆w2= η * ( Desired – Output) * xi = 1*(-1-1)*1 =-2
w2new = ∆w2+ wold = -2 +2 = 0  go to next sample
W2 = 2
W1 = -2
W0 = 2
x2 = 1
x1 = 0
x0 = 1
y
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
Feed Forward Neural Network
19
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

20. Example :Single Layer Perceptron cont.
1. Take the third sample, and initialize all weights to zero:
2. Calculate Net = 𝒊=𝟎
𝒏
wixi = 0*1 + -2*-1 + 0*-1 = 2
f(Net) = 1
3. while f(net) = desired so keep weights as they are and go to next sample
W2 = 0
W1 = -2
W0 = 0
x2 = -1
x1 = -1
x0 = 1
y
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
Feed Forward Neural Network
20
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

21. Example :Single Layer Perceptron cont.
1. Take the fourth sample, and initialize all weights to zero:
2. Calculate Net = 𝒊=𝟎
𝒏
wixi = 0*1 + -2*0 + 0*-1 = 0
f(Net) = -1
3. while f(net) = desired so keep weights as they are and go to next sample
W2 = 0
W1 = -2
W0 = 0
x2 = -1
x1 = 0
x0 = 1
y
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
Feed Forward Neural Network
21
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

22. Example :Single Layer Perceptron cont.
1. Back again to the first sample, and initialize all weights to zero:
2. Calculate Net = 𝒊=𝟎
𝒏
wixi = 0*1 + -2*-1 + 0*1 = 2
f(Net) = 1
3. while f(net) = desired so keep weights as they are and go to next sample
W2 = 0
W1 = -2
W0 = 0
x2 = 1
x1 = -1
x0 = 1
y
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
Feed Forward Neural Network
22
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

23. Example :Single Layer Perceptron cont.
1. Back again to the first sample, and initialize all weights to zero:
2. Calculate Net = 𝒊=𝟎
𝒏
wixi = 0*1 + -2*0 + 0*1 = 0
f(Net) = -1
3. while f(net) = desired so keep weights as they are and go to next sample
so the weighs used for this set is
W2 = 0
W1 = -2
W0 = 0
x2 = 1
x1 = 0
x0 = 1
y
x2
X1
x0
1
1
-1
1
-1
1
0
1
1
-1
-1
1
-1
-1
0
1
W2 = 0
W1 = -2
W0 = 0
Feed Forward Neural Network
23
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

24. Example :Single Layer Perceptron cont.
3. (-2, 0) = 𝐢=𝟎
𝐧
wixi = 0*1 + -2*-2 + 0*0 = 4
f(Net) = 1 so it is classified as class 1
(1, 1) = 𝐢=𝟎
𝐧
wixi = 0*1 + 1*-2 + 1*0 = -2
f(Net) = 1 so it is classified as class -1
(0, 1) = 𝐢=𝟎
𝐧
wixi = 0*1 + 0*-2 + 0*1 = 0
f(Net) = 1 so it is classified as class -1
(-1, -2) = 𝐢=𝟎
𝐧
wixi = 0*1 + -2*-1 + 0*-2 = 2
f(Net) = 1 so it is classified as class 1
Feed Forward Neural Network
24
F(Net) =
1 𝑁𝑒𝑡 > 0
−1 𝑁𝑒𝑡 ≤ 0

25. Perceprton implementation using
matlab
Feed Forward Neural Network
25

26. Simple Run using previous code
Step 1 : initialize input
Step 2 : initialize output
Step 3 : Call the function
Step 4 : testing sample
Feed Forward Neural Network
26

27. The End
Feed Forward Neural Network
27