Artificial Neural Networks_Bioinsspired_Algorithm

1. Artificial Neural Networks
and Bio-inspired Algorithms
by
Dr. Venkatanareshbabu K
Dept. of CSE,
Assistant Professor,
NIT Goa
1

2. Introduction
• Bio-inspired computing represents the umbrella of different studies of
computer science, mathematics, and biology in the last years.
• Bio-inspired computing optimization algorithms is an emerging approach
which is based on the principles and inspiration of the biological evolution of
nature to develop new and robust competing techniques.
• In the last years, the bio-inspired optimization algorithms are recognized in
machine learning to address the optimal solutions of complex problems in
science and engineering.

3. Contd...
• A genetic algorithm (GA) is a method for solving both constrained
and unconstrained optimization problems based on a natural
selection process that mimics biological evolution. The algorithm
repeatedly modifies a population of individual solutions. At each
step, the genetic algorithm randomly selects individuals from the
current population and uses them as parents to produce the children
for the next generation. Over successive generations, the population
"evolves" toward an optimal solution.
• You can apply the genetic algorithm to solve problems that are not
well suited for standard optimization algorithms, including problems
in which the objective function is discontinuous, nondifferentiable,
stochastic, or highly nonlinear.

4. Genetic Algorithm
• 1) Concept of genetic algorithm.
• 2) Fitness function
• 3) Crossover
• 4) Mutation
4

5. Genetic Algorithms
• Basics are discussed below:
• Population: It is a subset of all encoded solutions to the given
problem.
• Chromosomes: A chromosome is one such solution to the
given problem.
• Gene: It is a one element position of a chromosome.
• Allele: It is the value of a gene takes for a particular
chromosome.
5

6. Chromosome-Modelling
6

7. Contd…
• Genotype: Genotype is the population in the computation
space. The solutions are represented in a easy way and
manipulated using a computation system.
• Phenotype: Phenotype is the population in real world solution
space in which solutions are represented in a way they are
represented in real world solutions.
• Genetic Operations: These alter the genetic composition of the
offspring. These include : Crossover, Mutation, Selection, etc.
7

8. Encoding & Decoding
8

9. Algorithm
Algorithm:
• Population initialization:
• Loop until termination criteria reached
• {
• Fitness function calculation
• Crossover
• Mutation
• Survivor selection
• }
• Terminate and return best.
9

10. Crossover Operations
10

11. Crossover_examples

12. Mutation Operations
12

13. Illustrations
13

14. Step-3
14

15. Step-4
15

16. 16

17. 17

18. 18
1
2
3
4

19. 19

20. 20

21. 21

22. 22

23. PSO
• PSO is a stochastic optimization technique
developed by Dr. Eberhart and Dr.
Kennedy in 1995, inspired by social
behavior of bird flocking or fish schooling.

24. PSO

25. Algorithm

26. Similarities with PSO and GA

27. PSO-Code

29. PSO-Example

30. PSO-Example-Epoch1

31. Example-PSO-Epoch1

32. PSO-Epoch2

33. PSO-Epoch3

34. Artificial Neural Networks
34

35. 35
Artificial Neuron and Biological Neuron[1]

36. 36
Artificial Neuron and Biological Neuron[2]

37. 8/26/2022 37
2. Back Propagation(BP)
1. Perceptron
3. Autoencoder
4. Deep Neural Network(DNN)
5. DNN implementation
Contents

38. 8/26/2022 38
Let us learn Perceptron Algorithm

39. 8/26/2022 39

40. 40

41. 8/26/2022 41
x1 x2 W11 W21 Comp
uted
Target delta ΔW11 ΔW21
0 0 0.1 0.3 0 0 0 0 0
0 1 0.1 0.3 0 1 1 0 0.2
1 0 0.1 0.5 0 1 1 0.2 0
1 1 0.3 0.5 1 1 0 0 0
Epoch 1
‘OR’ Problem
η=0.2= Learning rate
θ= 0.5= Threshold
x1 x2 Target
0 0 0
0 1 1
1 0 1
1 1 1

42. 8/26/2022 42
x1 x2 W11 W21 Comp
uted
Target delta ΔW11 ΔW21
0 0 0.3 0.5 0 0 0 0 0
0 1 0.3 0.5 0 1 1 0 0.2
1 0 0.3 0.7 0 1 1 0.2 0
1 1 0.5 0.7 1 1 0 0 0
Epoch 2

43. 8/26/2022 43
x1 x2 W11 W21 Comp
uted
Target delta ΔW11 ΔW21
0 0 0.5 0.7 0 0 0 0 0
0 1 0.5 0.7 1 1 0 0 0
1 0 0.5 0.7 0 1 1 0.2 0
1 1 0.7 0.7 1 1 0 0 0
Epoch 3

44. 8/26/2022 44
x1 x2 W11 W21 Comp
uted
Target delta ΔW11 ΔW21
0 0 0.7 0.7 0 0 0 0 0
0 1 0.7 0.7 1 1 0 0 0
1 0 0.7 0.7 1 1 0 0 0
1 1 0.7 0.7 1 1 0 0 0
Epoch 4

45. 45
X1 X2 X3 Target
0 0 0.1 0
0.9 1 0.6 1
1 0 0.2 0
1 1 0.8 1
How do you construct perceptron for the following data?
0.5
0.3
0.7

46. 8/26/2022 46
Perceptron Algorithm works only for linearly separable
problem
Why?

47. 8/26/2022 47

48. 8/26/2022 48

49. 49
What is bias in a neural network?
The activation function in Neural Networks takes an input 'x' multiplied
by a weight 'w'. Bias allows you to shift the activation function by
adding a constant (i.e. the given bias) to the input. Bias in Neural
Networks can be thought of as analogous to the role of a constant in a
linear function, whereby the line is effectively transposed by the
constant value.

50. 50
X1 X2 bias Target
0 0 1 0
0 1 1 1
1 0 1 1
1 1 1 1
1
0.5
0.3
0.7
Bias in a neural network
L1
L2

51. 8/26/2022 51
Let us learn BPAlgorithm

52. 52

53. 8/26/2022 53
x1 x2 Target
0.4 -0.7 0.1
0.3 -0.5 0.05
0.6 0.1 0.3
0.2 0.4 0.25
0.1 -0.2 0.12
Sample dataset

54. 54
(Used in Perceptron)
(Used in Back Propagation)

55. 8/26/2022 55

56. 8/26/2022 56
8/26/2022 56
Weight Updating Equations:
Input to hidden :
hidden to output :

57. 8/26/2022 57
O1= 0.47
δ1 = 0.005
O2= 0.45
δ2 = -0.0083

58. 58

59. 59

60. 60
Bias in Back Propagation?

61. 61
Autoencoder
1. Input and Output contains same number of features.
Fig: Architecture of Autoencoder

62. 62
Dimensionality Reduction
A. Feature Selection
B. Feature Extraction
X1 X2 X3 X4 X5 X6
0.1 0.8 0.6 0.2 0.1 0.4
0.2 0.1 0.3 0.9 0.2 0.4
0.3 0.6 0.7 0.2 0.3 0.2
Y1 Y2 Y3
0.1 0.7 0.9
0.2 0.6 0.1
0.1 0.3 0.2
X1 X2 X3 X4 X5 X6
0.1 0.8 0.6 0.2 0.1 0.4
0.2 0.8 0.6 0.9 0.5 0.4
0.3 0.8 0.6 0.2 0.8 0.2
x1 x4 x5
0.1 0.2 0.1
0.2 0.9 0.5
0.3 0.2 0.8

63. 63
X1 X2 X3 X4 X5 X6
0.1 0.8 0.6 0.2 0.1 0.4
0.2 0.1 0.3 0.9 0.2 0.4
0.3 0.6 0.7 0.2 0.3 0.2
Y1 Y2 Y3
0.1 0.7 0.9
0.2 0.6 0.1
0.1 0.3 0.2

64. 64
Deep Neural Network

65. 65
65
Deep Neural Network
1. Stack of Autoencoders.

66. References
• 1. Towardsdatascience.com
• 2. https://www.ee.co.za/article/application-of-machine-learning-
algorithms-in-boiler-plant-root-cause-analysis.html
• 3. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, et al., Greedy
layer-wise training of deep networks, in: Advances in neural
information processing systems, 2007, pp. 153–160.
66

67. Thank You
67