A/nn kuhkj g gjh

1. Dr. Syed Sajjad Hussain Rizvi

2. 1. Motivation
2. What are Neural Networks?
3. Why are Artificial Neural Networks Worth Studying?
4. Learning in Neural Networks
5. Artificial Neural Networks compared with Classical AI
6. Some Current Artificial Neural Network Applications
7. Organization of the Nervous System and Brain
8. Brains versus Computers: Some Numbers
9. Biological Neurons and Neural Networks
10. The McCulloch-Pitts Neuron

3. Good at Not so good at
Rule-based systems:
doing what the programmer
wants them to do
Dealing with noisy data
Dealing with unknown
environment data
Massive parallelism
Fault tolerance
Adapting to circumstances

4.  Databases
 Information Systems
 Classical game programming
 Programmed autonomous processes
 Data analysis (financial, statistical, and probabilistic)
 Devices drivers
 Websites (static or dynamic)
 Basic operating system
 Mobile Applications
 Graphical Modeling tools (AutoCAD, Maya, 3D studio
max etc.)
 Office Automation Tools
 Rule based system

5.  Dynamic Routes in Routers
 Medical Imaging systems
 Voice Identification
 Image retrieval
 Computer Vision
 DSS/ ES ( e-doctor, student performance prediction)
 Semantic Extraction from text and multimedia
documents (data mining)
 Semantic Websites (LinkedIn, facebook, BBC sports
page world cup 2010).
 Knowledge representation and reasoning systems
(Anomaly detection, self healing systems etc.)
 GIS semantic extraction

7. 1) Hard computing, i.e., conventional computing, requires a precisely stated
analytical model and often a lot of computation time. Soft computing differs
from conventional (hard) computing in that, unlike hard computing, it is
tolerant of imprecision, uncertainty, partial truth, and approximation. In
effect, the role model for soft computing is the human mind.
2) Hard computing based on binary logic, crisp systems, numerical analysis
and crisp software but soft computing based on fuzzy logic, neural nets and
probabilistic reasoning.
3) Hard computing requires programs to be written; soft computing can evolve
its own programs
4) Hard computing uses two-valued logic; soft computing can use multivalve
or fuzzy logic
5) Hard computing is deterministic; soft computing incorporates stochasticity
6) Hard computing requires exact input data; soft computing can deal with
ambiguous and noisy data
7) Hard computing is strictly sequential; soft computing allows parallel
computations
8) Hard computing produces precise answers; soft computing can yield
approximate answers

8. Soft Computing
Classification
Artificial Neural
Network (ANN)
Single Layer
Perceptron
Multilayer
Perceptron
Radial Basis
function
Self organizing
Map
Support Vector
Machine
Fuzzy Logic
System (FLS)
Optimization
Genetic Algo
Ant colony
Tabo

9. 1. Neural Networks (NNs) are networks of neurons, for example, as
found in real (i.e. biological) brains.
2. Artificial Neurons are basic approximations of the neurons found
in brains. They may be physical devices, or purely mathematical
constructs.
3. Artificial Neural Networks (ANNs) are networks of Artificial
Neurons, and hence constitute basic approximations to parts of real
brains. They may be physical devices, or simulated on conventional
computers.
4. From a practical point of view, an ANN is just a parallel
computational system consisting of many simple processing
elements connected together in a specific way in order to perform a
particular task (classification).
5. One should never lose sight of how basic the approximations are,
and how over-simplified our ANNs are compared to real brains.

10. 1. They are extremely powerful computational method.
2. Massive parallelism makes them very efficient.
3. They can learn and generalize from training data – so there is no need for
enormous skills of programming.
4. They are particularly fault tolerant – this is equivalent to the “graceful
degradation” found In biological systems.
5. They are very noise tolerant – so they can cope with situations where
normal symbolic systems would have difficulty.

11. As with the field of AI in general, there are two basic goals for neural
network research:
Brain modeling :
The scientific goal of building models of how real brains work. This can
potentially help us understand the nature of human intelligence, formulate
better teaching strategies, or better remedial actions for brain damaged
patients.
Artificial System Building :
The engineering goal of building efficient systems for real world
applications. This may make machines more powerful, relieve humans of
tedious tasks, and may even improve upon human performance.

12. There are many forms of neural networks. Most operate by passing neural
‘activations’ through a network of connected neurons.
One of the most powerful features of neural networks is their ability to learn
and generalize from a set of training data. They adapt the strengths/weights
of the connections between neurons so that the final output activations are
correct.
There are three broad types of learning:
1. Supervised Learning (i.e. learning with a teacher)
2. Reinforcement learning (i.e. learning with limited feedback)
3. Unsupervised learning (i.e. learning with no help)

13. The distinctions can put under three headings:
1. Level of Explanation
2. Processing Style
3. Representational Structure
In practice, the distinctions are becoming increasingly blurred.

14. Brain modeling
Models of human development – help children with developmental
Problems– aid our understanding of how the brain works Neuropsychological
models – suggest remedial actions for brain damaged patients
Real world applications
Financial modeling – predicting stocks, shares, currency exchange rates
Other time series prediction – climate, weather, airline marketing tactician
Computer games – intelligent agents, backgammon, first person shooters
Control systems – autonomous adaptable robots, microwave controllers
Pattern recognition – speech recognition, hand-writing recognition, sonar signals
Data analysis – data compression, data mining, PCA, GTM
Noise reduction – function approximation, ECG noise reduction
Bioinformatics – protein secondary structure, DNA sequencing

15. The Nervous System:
The human nervous system can be broken down into three stages that may be
represented in block diagram form as:
 The receptors collect information from the environment – e.g. photons on the retina.
 The effectors generate interactions with the environment – e.g. activate muscles.
 The flow of information/activation is represented by arrows – feedforward and feedback.
 Naturally, in this module we will be primarily concerned with the neural network in the middle.

16. 1. There are approximately 10 billion neurons in the human cortex, compared with few
processors in the most powerful parallel computers.
2. Each biological neuron is connected to several thousands of other neurons, similar
to the connectivity in powerful parallel computers.
3. Lack of processing units can be compensated by speed. The typical operating
speeds of biological neurons is measured in milliseconds (10-3
s), while a silicon
chip can operate in nanoseconds (10-9
s).
4. The human brain is extremely energy efficient, using approximately 10-16
joules
per operation per second, whereas the best computers today use around 10-6
joules
per operation per second.
5. Brains have been evolving for tens of millions of years, computers have been
evolving for tens of decades.

17. 1. The majority of neurons encode their activations or outputs as a series of
brief electrical pulses (i.e. spikes or action potentials).
2. The neuron’s cell body (soma) processes the incoming activations and
converts them into output activations.
3. The neuron’s nucleus contains the genetic material in the form of DNA.
This exists in most types of cells, not just neurons.
4. Dendrites are fibres which emanate from the cell body and provide the
receptive zones that receive activation from other neurons.
5. Axons are fibres acting as transmission lines that send activation to other
neurons.
6. The junctions that allow signal transmission between the axons and
dendrites are called synapses. The process of transmission is by diffusion
of chemicals called neurotransmitters across the synaptic cleft.

19. This vastly simplified model of real neurons is also known as a Threshold
Logic Unit :
1. A set of synapses (i.e. connections) brings in activations from other neurons.
2. A processing unit sums the inputs, and then applies a non-linear activation function
(i.e. squashing/transfer/threshold function).
3. An output line transmits the result to other neurons.

20. A function y = f(x) describes a relationship (input-output mapping) from x to y.
Example 1 The threshold or sign function sgn(x) is defined as
Example 2 The logistic or sigmoid function Sigmoid(x) is defined as
This is a smoothed (differentiable) form of the threshold function.