Introduction to Neural
Hafiz Syed Adnan Ahmed
• Artificial Neural Network is based on the biological nervous
system as Brain
• It is composed of interconnected computing units called
• ANN like human, learn by examples
Why Artificial Neural Networks?
There are two basic reasons why we are interested in
building artificial neural networks (ANNs):
• Technical viewpoint: Some problems such as
character recognition or the prediction of future
states of a system require massively parallel and
• Biological viewpoint: ANNs can be used to
replicate and simulate components of the human
(or animal) brain, thereby giving us insight into
natural information processing.
Science: Model how biological neural
systems, like human brain, work?
• How do we see?
• How is information stored in/retrieved
• How do you learn to not to touch fire?
• How do your eyes adapt to the amount
of light in the environment?
• Related fields: Neuroscience,
Psychology, Psychophysiology, Cognitive
Science, Medicine, Math, Physics.
Real Neural Learning
• Synapses change size and strength with
• Hebbian learning: When two connected neurons
are firing at the same time, the strength of the
synapse between them increases.
• “Neurons that fire together, wire together.”
• Human brain = tens of thousands
• Each neuron is connected to
thousands other neurons
• A neuron is made of:
• The soma: body of the neuron
• Dendrites: filaments that provide
input to the neuron
• The axon: sends an output signal
• Synapses: connection with other
neurons – releases certain
quantities of chemicals called
neurotransmitters to other
Modelling a Neuron
W j , ia j
:Activation value of unit j
:Weight on the link from unit j to unit i
:Weighted sum of inputs to unit i
:Activation value of unit i
What is an artificial neuron ?
• Definition : Non linear, parameterized function with
restricted output range
How do NNs and ANNs work?
• Information is transmitted as a series of
electric impulses, so-called spikes.
• The frequency and phase of these spikes
encodes the information.
• In biological systems, one neuron can be
connected to as many as 10,000 other
• Usually, a neuron receives its information
from other neurons in a confined area
Navigation of a car
• Done by Pomerlau. The network takes inputs from a 34X36 video image
and a 7X36 range finder. Output units represent “drive straight”, “turn
left” or “turn right”. After training about 40 times on 1200 road
images, the car drove around CMU campus at 5 km/h (using a small
workstation on the car). This was almost twice the speed of any other
non-NN algorithm at the time.
Automated driving at 70 mph on a
into one out of
Computers vs. Neural Networks
fast processing units
The common type of ANN consists of three layers
of neurons: a layer of input neurons connected to
the layer of hidden neuron which is connected to
a layer of output neurons.
Architecture of ANN
• Feed-Forward networks
Allow the signals to travel one way from input to
• Feed-Back Networks
The signals travel as loops in the network, the
output is connected to the input of the network
Comparison of Brains and Traditional
• 200 billion neurons, 32
• Element size: 10-6 m
• Energy use: 25W
• Processing speed: 100 Hz
• Parallel, Distributed
• Fault Tolerant
• Learns: Yes
• 1 billion bytes RAM but
trillions of bytes on disk
• Element size: 10-9 m
• Energy watt: 30-90W (CPU)
• Processing speed: 109 Hz
• Serial, Centralized
• Generally not Fault Tolerant
• Learns: Some
Neural Networks (Applications)
• Face recognition
• Time series prediction
• Process identification
• Process control
• Optical character recognition
• Adaptative filtering
“If the brain were so simple
that we could understand it
then we’d be so simple that