Introduction to Soft Computing by Dr.S.Jagadeesh Kumar

INTRODUCTION TO
SOFT COMPUTING
By Dr.S.Jagadeesh Kumar
NIMS University, Jaipur

WHAT IS SOFT COMPUTING?
• Soft computing is an umbrella term used to describe types of algorithms that
produce approximate solutions to unsolvable high-level problems in
computer science.
• Soft computing is a branch of artificial intelligence that focuses on the design
of intelligent systems capable of solving complex, imprecise or ambiguous
problems.
• Soft computing includes innovations like machine learning, artificial neural
networks, genetic algorithms, fuzzy logic, and expert systems.
• Soft computing exploits the given tolerance for imprecision, partial truth, and
uncertainty for a particular problem.
2

WHAT IS HARD COMPUTING?
• Hard computing is the conventional approach that is used in computing and
needs accurately stated analytical model.
• Traditional hard-computing algorithms heavily rely on concrete data and
mathematical models to produce solutions to problems.
• Soft computing is the reverse of conventional computing that aims to
provide approximation and to find quick solutions to complex real-life
problems.
• Soft computing refers to a group of computational techniques that are based
on artificial intelligence and natural selection. It provides cost-effective
solutions to the complex real-life problems for which hard computing
solution does not exist.
3

INTRODUCTION TO ARTIFICIAL INTELLIGENCE (1)
• Artificial intelligence (AI) is technology that enables computers and
machines to simulate human intelligence and problem-solving capabilities.
• Specific applications of AI include expert systems, natural language
processing, speech recognition and machine vision.
• AI has become a catchall term for applications that perform complex tasks
that once required human input, such as communicating with customers
online or playing chess. The term is often used interchangeably with its
subfields, which include machine learning (ML) and deep learning.
• AI augments human intelligence with rich analytics and pattern prediction
capabilities to improve the quality, effectiveness, and creativity.
4

INTRODUCTION TO ARTIFICIAL INTELLIGENCE (2)
• Artificial intelligence is the science of making machines that can think like
humans. It can do things that are considered “smart”.
• AI technology can process large amounts of data in ways, unlike humans.
The goal for AI is to be able to do things such as recognize patterns, make
decisions, and judge like humans.
• John McCarthy is considered as the father of Artificial Intelligence. John
McCarthy was an American computer scientist. The term "artificial
intelligence" was coined by him.
• AI is predicted to grow increasingly pervasive as technology develops,
revolutionising sectors including healthcare, banking, and transportation.
5

ARTIFICIAL NEURAL NETWORKS (1)
• An artificial neural network (ANN) is an interconnected group of nodes,
inspired by a simplification of neurons in a brain.
• Each circular node represents an artificial neuron and an arrow represents a
connection from the output of one artificial neuron to the input of another.
• A neural network has input layer(s), hidden layer(s), and output layer(s). It
can make sense of patterns, noise, and sources using an activation function.
• Artificial neural networks are trained using training sets. Suppose you want to
teach an ANN to recognize a cat. Then it is shown thousands of different
images of cats so that the network can learn to identify a cat.
6

7

• Artificial neural networks (ANNs) are biologically inspired computer
programs designed to simulate the way in which the human brain processes
information.
• ANNs gather their knowledge by detecting the patterns and relationships in
data and learn (or are trained) through experience, not from programming.
• Artificial neural networks are used for a range of applications, including
image recognition, speech recognition, machine translation, and medical
diagnosis.
• The fact that ANN learns from sample data sets is a significant advantage.
The most typical application of ANN is for random function approximation.
8

• Artificial Neural Networks are part of supervised machine learning which can
be used for solving both regression and classification problems.
• The common ANN types are: Multi-Layer Perceptrons (MLP), Convolutional
Neural Networks (CNN) and Recurrent Neural Networks (RNN).
• ANNs have the ability to learn and model non-linear and complex
relationships, which is really important because in real-life, many of the
relationships between inputs and outputs are non-linear as well as complex.
• In robotics, ANN allows the rover to plan and execute collision-free motions
within its environment and to reach its goal location while avoiding obstacles
and dangerous cosmic objects.
9

FUZZY SYSTEMS
• Fuzzy logic is a form of many-valued logic in which the truth value of
variables may be any real number between 0 and 1.
• It is employed to handle the concept of partial truth, where the truth value
may range between completely true and completely false.
• Fuzzy systems are structures based on fuzzy techniques oriented towards
information processing, where the usage of classical sets theory and binary
logic is impossible or difficult.
• Fuzzy logic is used in various fields, such as automotive systems, environment
control, and domestic goods. (The altitude control of spacecraft and
satellite, controlling the speed and traffic in the automotive systems)
10

GENETIC ALGORITHM AND
EVOLUTIONARY PROGRAMMING (1)
• Algorithms that follow laws of evolution are called "Evolutionary
algorithms“(EA). There are two sub-classes of EA: (1) Genetic Algorithm (GA)
that uses crossover, along with mutation as GA operators (2) Evolutionary
Programming, that uses only mutation as its operator.
• According to the law of evolution theory, the less fit are eliminated and the
fittest survive.
• The five important laws of evolution are: Variation, Inheritance, Selection,
Time and Adaptation (abbreviated here as VISTA).
• A genetic or evolutionary algorithm applies the principles of evolution found
in nature to the problem of finding an optimal solution to a solver problem.
11

GENETIC ALGORITHM AND
EVOLUTIONARY PROGRAMMING (2)
• Mutation: Inspired by the role of mutation of an organism's DNA in natural
evolution. An evolutionary algorithm periodically makes random changes or
mutations in one or more members of the current population, yielding a new
candidate solution (which may be better or worse than existing population
members).
• Crossover: Inspired by the role of sexual reproduction in the evolution of
living things. An evolutionary algorithm attempts to combine elements of
existing solutions in order to create a new solution, with some of the features
of each "parent." The elements (e.g. decision variable values) of existing
solutions are combined in a "crossover" operation, inspired by the crossover
of DNA strands that occurs in reproduction of biological organisms.
12

SWARM INTELLIGENT SYSTEMS (1)
• Swarm intelligence is defined as a collective behavior of a decentralized or
self-organized system. These systems consist of numerous individuals with
limited intelligence interacting with each other based on simple principles.
• Examples in natural systems of swarm intelligence include: Clustering
Behavior of Ants, Nest Building Behavior of Wasps and Termites, Flocking and
Schooling in Birds and Fish, Ant Colony Optimization, Particle Swarm
Optimization.
• Swarm intelligence (SI) is a subfield of artificial intelligence (AI) based on the
study of decentralized systems. These systems are typically made up of a
large number of simple agents that interact with each other and their
environment in order to accomplish a common goal.
13

SWARM INTELLIGENT SYSTEMS (2)
• Swarm Intelligence simulations can be used to study the dynamics of
complex systems such as social networks, economic markets, and
ecological ecosystems. Swarm Intelligence is used for simulation purposes in
order to understand how large groups of people behave and make
decisions.
• The benefits of using swarm intelligence include: improved analytic,
predictive, and better decision-making capabilities.
• The advantages of swarm intelligence are distribution, robustness, indirect
communication, and simplicity. The main disadvantage of swarm
intelligence optimization algorithm is that it is prone to premature
convergence and poor local optimization ability.
14

MCCULLOCH PITT'S MODEL OF ANN (1)
• The McCulloch Pitt's Model of Neuron is the earliest logical simulation of a
biological neuron, developed by Warren McCulloch and Warren Pitts in 1943.
• The motivation behind the McCulloh Pitt’s Model is a biological neuron. A
biological neuron takes an input signal from the dendrites and after
processing it passes onto other connected neurons as the output if the signal
is received positively, through axons and synapses. This is the basic working
of a biological neuron which is interpreted and mimicked using the
McCulloh Pitt’s Model.
• Dendrite: Receives signals from other neurons, Soma: Processes the
information, Axon: Transmits the output of this neuron, Synapse: Point of
connection to other neurons.
15

16

• McCulloch Pitt’s model of neuron is a fairly simple model which consists of
some (n) binary inputs with some weight associated with each one of them.
An input is known as ‘inhibitory input’ if the weight associated with the input is
of negative magnitude and is known as ‘excitatory input’ if the weight
associated with the input is of positive magnitude. As the inputs are binary,
they can take either of the 2 values, 0 or 1.
• We have a summation junction that aggregates all the weighted inputs and
then passes the result to the activation function. The activation function is a
threshold function that gives out 1 as the output if the sum of the weighted
inputs is equal to or above the threshold value and 0 otherwise.
17

• let’s say we have n inputs = { X1, X2, X3, …. , Xn }
• And we have n weights for each = {W1, W2, W3, …., W4}
• So the summation of weighted inputs X.W = X1.W1 + X2.W2 + X3.W3 +....+
Xn.Wn
If X ≥ ø(threshold value)
Output = 1
Else
Output = 0.
18

• Let’s Take a real-world example:
• A bank wants to decide if it can sanction a loan or not. There are 2
parameters to decide - Salary and Credit Score.
• So there can be 4 scenarios to assess - High Salary and Good Credit Score,
High Salary and Bad Credit Score, Low Salary and Good Credit Score, Low
Salary and Bad Credit Score.
• Let X1 = 1 denote high salary and X1 = 0 denote Low salary and X2 = 1
denote good credit score and X2 = 0 denote bad credit score.
19

• Let the threshold value be 2. The truth table is as
20
X1 X2 X1+X2 Loan approved
1 1 2 1
1 0 1 0
0 1 1 0
0 0 0 0

HEBBIAN AND DELTA LEARNING RULES (1)
• Learning rule enhances the Artificial Neural Network’s performance by
applying these rules over the network.
• Learning rule updates the weights and bias levels of a network when certain
conditions are met in the training process. It is a crucial part of the
development of the Neural Network.
• A learning rule in ANN is nothing but a set of instructions or a mathematical
formula that helps in reinforcing a pattern, thus improving the efficiency of a
neural network. There are 6 such learning rules that are widely used by
neural networks for training.
21

22

• Developed by Donald Hebb in 1949, Hebbian learning is an unsupervised
learning rule that works by adjusting the weights between two neurons in
proportion to the product of their activation.
• According to this rule, the weight between two neurons is decreased if they
are working in opposite directions and vice-versa. However, if there is no
correlation between the signals, then the weight remains the same. As for the
sign a weight carries, it is positive when both the nodes are either positive or
negative. However, in case of a one node being either positive or negative,
the weight carries a negative sign.
• Δw = αxiy (where Δw is the change in weight, α is the learning rate, xi is the
input vector and, y is the output)
23

• Developed by Bernard Widrow and Marcian Hoff, Delta learning rule is a
supervised learning rule with a continuous activation function.
• The main aim of this rule is to minimize error in the training patterns and thus,
it is also known as the least mean square method. The principle used here is
that of an infinite gradient descent and the changes in weight is equal to the
product of the error and the input.
• Δw_ij = η * (d_j - y_j) * f'(h_j) * x_i
(where ∆w_ij is the changes in weight between the ith input neuron and
jth output neuron, η is the learning rate, d_j is the target output, y_j is the actual
output, f'(h_j) is the derivative of the activation function, x_i is the ith input)
24

PERCEPTRON NETWORK (1)
• Perceptron network is a type of artificial neural network, which is a
fundamental concept in machine learning.
• The perceptron is a mathematical model of a biological neuron. While in
actual neurons the dendrite receives electrical signals from the axons of
other neurons; in the perceptron these electrical signals are represented as
numerical values.
• Artificial Neural Networks (ANN) and MultiLayer Perceptron (MLP) are both
types of neural networks used in machine learning. The main difference
between the two is that MLP is a type of ANN with specific architecture. ANN
is a computational model inspired by the biological neural networks present
the human brain.
25

• A Perceptron is an artificial neuron, and thus a neural network unit. It
performs computations to detect features or patterns in the input data. It is
an algorithm for supervised learning of binary classifiers. It is this algorithm
that allows artificial neurons to learn and process features in a data set.
• There are two types of perceptrons: Single layer perceptrons - These can
only learn linearly separable patterns. Multilayer perceptrons - These have
the greatest processing power. They are a class of feedforward neural
networks.
• The perceptron model is a more general computational model than
McCulloch-Pitts. It takes an input, aggregates it (weighted sum) and returns 1
only if the aggregated sum is more than some threshold else returns 0.
26

27

ADALINE NETWORK (1)
• ADALINE is a supervised learning model used to perform binary classification
and linear regression. The neural network consists of an input layer, an output
layer and a feedback layer that adjusts the weights of the input layer
according to the output obtained.
• Perceptron updates the weights for each row of the data whereas Adaline
takes in the whole data and updates the weights for the whole data. It does
not update weights for each rows.
• ADALINE was initially applied to problems like speech and pattern
recognition, but the main application of the ADALINE are adaptive filtering
and adaptive signal processing.
28

ADALINE NETWORK (2)
• Adaline's linear activation function implies that f(z)=z, which is a superfluous
step from a classification perspective (the output of this function is a
continuous variable and the output expected for a classification problem is
a categorical variable).
• With two inputs, a single adaline can realize 14 of the 16 possible binary logic
functions. The two it cannot learn are exclusive OR and exclusive NOR
functions. With many inputs, however, only a small fraction of all possible
logic functions are realizable, i.e., linearly separable.
• In Adaline, the output values are bipolar (+1,-1). Weights between the input
unit and output unit are adjustable. It uses the delta rule.
29

ADALINE NETWORK (3)
30

MADALINE NETWORK (1)
• MADALINE (Many ADALINE) is a three-layer (input, hidden, output), fully
connected, feed-forward artificial neural network architecture for
classification that uses ADALINE units in its hidden and output layers, i.e. its
activation function is the sign function.
• Adaline doesn't have any hidden layer, and Madaline has one hidden layer.
• The Madaline (supervised Learning) model consists of many Adaline in
parallel with a single output unit. The Adaline layer is present between the
input layer and the Madaline layer hence Adaline layer is a hidden layer.
The weights between the input layer and the hidden layer are adjusted, and
the weight between the hidden layer and the output layer is fixed.
31

• It uses the majority vote rule, the output would have an answer either true or
false. Adaline and Madaline layer neurons have a bias of ‘1’ connected to
them. Use of multiple Adaline helps counter the problem of non-linear
separability.
• In Madaline, the third layer is the output layer, the weights between hidden
and output layer is fixed they are not adjustable.
• Madaline stands for Multiple Adaptive Linear Neuron, is a network which
consists of many Adalines in parallel. It will have a single output unit. The
Madaline is a multilayer extension of the single-neuron bipolar Adaline to a
network.
32

33

Introduction to Soft Computing by Dr.S.Jagadeesh Kumar

Recommended

Recommended

More Related Content

Similar to Introduction to Soft Computing by Dr.S.Jagadeesh Kumar

Similar to Introduction to Soft Computing by Dr.S.Jagadeesh Kumar (20)

Recently uploaded

Recently uploaded (20)

Introduction to Soft Computing by Dr.S.Jagadeesh Kumar