Artificial Neural Networks used for learning algorithms in the field of Artificial Intelligence. BS(CS) level, semester 5th slides for the course Artificial Intelligence in Federal Urdu University of Arts Science and Technology.

1. Artificial Neural Networks
application of AI
Sir Hafiz Syed Muhammad Rafi
Federal Urdu University of Arts Science and
Technology (FUUAST)
11/7/2019

2. Background
- Neural Networks can be :
- Biological models
- Artificial models
- Desire to produce artificial systems capable of
sophisticated computations similar to the human brain.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

3. Biological analogy and some main ideas
• The brain is composed of a mass of interconnected neurons
– each neuron is connected to many other neurons
• Neurons transmit signals to each other
• Whether a signal is sent, depends on the strength of the
bond (synapse) between two neurons
In more description, a brain has great structure and the
ability to build up its own rules through what we usually
refer to as “experience”, (hard-wiring)
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

4. How Does the Brain Work ? (1)
NEURON
- The cell that performs information processing in the brain.
- Fundamental functional unit of all nervous system tissue.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

5. Each consists of :
SOMA, DENDRITES, AXON, and SYNAPSE.
How Does the Brain Work ? (2)
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

6. Biological inspiration
Dendrites
Soma (cell body)
Axon
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

7. Biological inspiration
synapses
axon
dendrites
The information transmission happens at the synapses.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

8. Brain vs. Digital Computers
- Computers require hundreds of cycles to simulate
a firing of a neuron.
- The brain can fire all the neurons in a single step.
Parallelism
- Serial computers require billions of cycles to
perform some tasks but the brain takes less than
a second.
e.g. Face Recognition
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

9. Comparison of Brain and computer
Human Computer
Processing
Elements
100 Billion
neurons
10 Million
gates
Interconnects 1000 per
neuron
A few
Cycles per sec 1000 500 Million
2X
improvement
200,000
Years
2 Years
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

10. What is Artificial Neural Network?
A neural network is a massively parallel distributed
processor made up of simple processing units
(neurons), which has a natural propensity for storing
experiential knowledge.
(i) Knowledge is acquired by the network from its
environment through a learning process.
(ii) Interneuron connection strength, known as
synaptic weights, are used to store the acquired
knowledge.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

11. Neurons vs. Units
- Each element of NN is a node called unit.
- Units are connected by links.
- Each link has a numeric weight.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

12. Are current computer a wrong
model of thinking?
• Humans can’t be doing the sequential analysis
we are studying
– Neurons are a million times slower than gates
– Humans don’t need to be rebooted or debugged
when one bit dies.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

13. Standard structure of an artificial neural
network
• Input units
– represents the input as a fixed-length vector of numbers (user
defined)
• Hidden units
– calculate thresholded weighted sums of the inputs
– represent intermediate calculations that the network learns
• Output units
– represent the output as a fixed length vector of numbers
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

14. Biological and Artificial Neuron
soma
synapse
dendrite
axon
Structure of Biological
neuron
 k
y( )f
1k
w
2k
w
km
w
k
b
1
x
mx
2
x
1
net
Structure of Artificial
neuron
( )ky f net
1
m
ki i k
i
net w x b

 
Activation
function
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

15. How NN learns a task.
Issues to be discussed
- Initializing the weights.
- Use of a learning algorithm.
- Set of training examples.
- Encode the examples as inputs.
- Convert output into meaningful results.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

16. Simple Computations in this network
-There are 2 types of components: Linear and
Non-linear.
- Linear: Input function
- calculate weighted sum of all inputs.
- Non-linear: Activation function
- transform sum into activation level.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

17. Activation Function
y
x
( )y f x x 
1
y
x
2
2( ) 1
1 x
y f x
e
  

1
y
x
1( )
1 xy f x
e 

Any continuous (differentiable) function can be used as an activation
function in a neural network.
The nonlinear behavior of the neural networks is inherited from the
used nonlinear activation functions.
y
x
2
( ) axy f x e 
Linear
function
Tangent
sigmoid
function
Logarithmic
sigmoid
function
Radial basis
function
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

18. Network Architectures
Single layer feedforward network
(Single layer perceptron)
Ex: working on ON, AND operator
Input
layer
Output
layer
Multilayer feedforward network
(Multilayer perceptron)
Ex: curve fitting (fitnet )and
pattern recognition (patternnet)
Input
layer
Output
layer
Hidden
layer11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

19. Network Architectures
1
z
1
z
1
z
1
z
Diagonal recurrent networks
Input
layer
Output
layer
Hidden
layer
Input
layer
Output
layer
Hidden
layer
Delay element in a
recurrent network
Fully recurrent networks
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

20. (1)Perceptrons
- First studied in the late 1950s.
- Also known as Layered Feed-Forward Networks.
- The only efficient learning element at that time was
for single-layered networks.
- Today, used as a synonym for a single-layer,
feed-forward network.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

21. Perceptrons
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

22. Node biases
• A node’s output is a weighted function of its
inputs
• What is a bias?
• How can we learn the bias value?
• Answer: treat them like just another weight
( )ky f net
1
m
ki i k
i
net w x b

  Formula used in the calculation of Perceptron, here
Sum of all inputs and weights are multiplied and add
to their respective bias. It gives result to activation
function and then it transform in to output of that
particular input.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

23. Use of OR and AND in
Perceptron
• OR function AND function
– Now use toolbox of Neural Network in Matlab
(nntool) and apply above functions.
Input1 Input2 Output
0 0 0
0 1 1
1 0 1
1 1 1
Input1 Input2 Output
0 0 0
0 1 0
1 0 0
1 1 1
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

24. Perceptrons & XOR
• XOR function
Input1 Input2 Output
0 0 0
0 1 1
1 0 1
1 1 0
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

25. Multi-Layer
Neural Nets
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

26. Feed Forward
Networks
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

27. Feed-forward NN with hidden layer
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

28. Feed-forward Networks
- Arranged in layers.
- Each unit is linked only in the unit in next layer.
- No units are linked between the same layer, back to
the previous layer or skipping a layer.
- Computations can proceed uniformly from input to
output units.
- No internal state exists.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

29. Difference between Multi-layer Networks
and Perceptrons
- Have one or more
layers of hidden units.
- With two possibly very
large hidden layers, it is
possible to implement
any function.
- Networks without hidden
layer are called perceptrons.
- Perceptrons are very limited
in what they can represent,
but this makes their learning
problem much simpler.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

30. 11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)
-Feed-forward Networks can be divided into two
categories if you using MATLAB Neural Network toolbox,
You can enter in it by using nnstart,
(1) Curve fitting (fitnet)
It is the process of constructing a curve,
or mathematical function, that has the best fit to a
series of data points, possibly subject to constraints.
(2) Pattern Recognition (Patternnet)
Pattern Recognition is the assignment of a label to a
given input value. An example of pattern recognition
is classification, which attempts to assign each input
value to one of a given set of classes.
Feed-forward Networks

31. Supervised Learning
 Supervised learning which incorporates an external
teacher, so that each output unit is told what its desired
response to input signals ought to be.
 An important issue concerning supervised learning is the
problem of error convergence, i.e the minimization of
error between the desired and computed unit values.
 The aim is to determine a set of weights which minimizes
the error. One well-known method, which is common to
many learning paradigms is the least mean square (LMS)
convergence.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

32. Supervised Learning
 In this sort of learning, the human teacher’s experience is
used to tell the NN which outputs are correct and which
are not.
 This does not mean that a human teacher needs to be
present at all times, only the correct classifications
gathered from the human teacher on a domain needs to
be present.
 The network then learns from its error, that is, it changes
its weight to reduce its prediction error.
 Application:
We use fitnet and patternet function in this sort.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

33. Unsupervised Learning
 Unsupervised learning uses no external teacher and is based
upon only local information. It is also referred to as self-
organization, in the sense that it self-organizes data presented
to the network and detects their emergent collective
properties.
 The network is then used to construct clusters of similar
patterns.
 This is particularly useful is domains were a instances are
checked to match previous scenarios. For example, detecting
credit card fraud.
We use it in Clustering and Self organization map functions.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

34. Neural Network in Use
Since neural networks are best at identifying patterns or trends
in data, they are well suited for prediction or forecasting
needs including:
– sales forecasting
– industrial process control
– customer research
– data validation
– risk management
ANN are also used in the following specific paradigms: recognition of
speakers in communications; diagnosis of hepatitis; undersea mine
detection; texture analysis; three-dimensional object recognition;
hand-written word recognition; and facial recognition.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

35. Neural networks in Medicine
 Artificial Neural Networks (ANN) are currently a 'hot'
research area in medicine and it is believed that they will
receive extensive application to biomedical systems in the
next few years.
 At the moment, the research is mostly on modeling parts
of the human body and recognizing diseases from various
scans (e.g. cardiograms, CAT scans, ultrasonic scans, etc.).
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)

36. Continued
 Neural networks are ideal in recognizing diseases using
scans since there is no need to provide a specific
algorithm on how to identify the disease.
 Neural networks learn by example so the details of how
to recognize the disease are not needed.
 What is needed is a set of examples that are
representative of all the variations of the disease.
 The quantity of examples is not as important as the
'quality'. The examples need to be selected very carefully
if the system is to perform reliably and efficiently.
11/7/2019
Federal Urdu University of Arts Science and
Technology (FUUAST)