Neuroevolution is a technique that uses genetic algorithms to train neural networks. It evolves network topologies, weights, and hyperparameters without relying on backpropagation or large labeled training datasets. The NEAT algorithm is introduced as a prominent neuroevolution method. It uses speciation to protect innovative network topologies and incremental evolution. NEAT encodes networks and uses historical markings to solve the competing conventions problem during crossover. Extensions to NEAT aim to further improve the scalability and efficiency of neuroevolution.

1. Building Neural Network
Through Neuroevolution
Alexandre Bergel
DCC - University of Chile
http://bergel.eu
10/12/2019
1

2. 2
In a Nutshell
Classical deep learning is about training models
Neuroevolution is about evolving neural
networks
Neuroevolution is a combination of genetic
algorithm and deep learning

3. 3
Outline
Genetic Programming: Simulation of Evolution
Neuroevolution: Combining Deep Learning and
Evolution
NEAT Algorithm: Evolving species of networks

4. 4

5. 5
Premise of Genetic Algorithm
Natural selection is pioneered by Charles Darwin
(1809-1882), biologist who worked on natural evolution
“On the Origin of Species by Means of Natural
Selection, or the Preservation of Favoured Races in the
Struggle for Life”, 1859
“One general low, leading to the advancement of all
organic beings, namely, vary, let the strongest live
and the weakest die”
— Charles Darwin

6. 6
“Guess the 3-letter word I have in mind”
Secret word: cat

7. 7
“Guess the 3-letter word I have in mind”
“For each try, I tell you the number of correct letters”
Secret word: cat

8. 8
Secret word: cat
“cow”
“poz”
“gaz”
1
0
1

9. 9
gaz
cow
poz

10. 10
Using genetic
operators,
the words are
combined and
some letters are
randomly modified
gaz
cow
gow
caz
poz pow

11. 11
atc
gozgaz
cow
gow
caz
poz pow poz

12. 12
gozgaz
cow
gow
caz
1st generation 2nd generation 3rd generation
atc
poz pow poz

13. Flow chart of an evolution algorithm
13
Selection
Population Parents
Offspring
Reproduction
Replacement

14. Flow chart of an evolution algorithm
14
Selection
Population Parents
Offspring
Reproduction
Replacement
Many EAs are around:
Ant colony optimization,
Artificial immune system,
Cultural algorithms,
Genetic Algorithm,
Genetic Programming,
Grammatical evolution,
…

15. Flow chart of a genetic algorithm
15
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End

16. Flow chart of a genetic algorithm
16
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End
We create a set of
random 3-letter words

17. Flow chart of a genetic algorithm
17
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End
Compute the score
of each word

18. Flow chart of a genetic algorithm
18
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End
Do we have a word with
score = 3?

19. Flow chart of a genetic algorithm
19
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End
We take the words that
are close to 3

20. Flow chart of a genetic algorithm
20
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End
We combine and mutate
words to form a new set of
words

21. Flow chart of a genetic algorithm
21
Yes
Initialize
population
Evaluate fitness
Solution
found?
Selection
Reproduction
No
End
We combine and mutate
words to form a new set of
words
DEMO

22. 22
Genetic Algorithm in a Nutshell
Evolutionary computation technique that
automatically solves problems without specifying the
form or structure of the solution in advance
Generally speaking, genetic algorithms are
simulations of evolution, using biological genetic
operations
Finding such as is maximal, is a tuple
of any arbitrary dimension and domain value,
x f(x) = y x
y ∈ ℝ

23. Flexible Muscle-Based Locomotion
for Bipedal Creatures
23
https://www.youtube.com/watch?v=pgaEE27nsQw

24. 24

25. 25

26. 26
= Neuroevolution
+

27. 27
Gradient descent
Gradient descent is commonly used to make a
neural network “learn”

28. 28
Gradient descent requires examples
Examples are required to train a neural network
Picture recognition may require thousands of
pictures
Requiring a large number of training examples
may be problematic
e.g., video games, self-driving cars

29. 29
Generalization
How do we get from where we are to artificial
general intelligence?
How to make the learning process closer to how
human learn?
https://www.nature.com/articles/s42256-018-0006-z

30. 30
Neuroevolution
“Neuroevolution is a form of artificial intelligence
that uses evolutionary algorithms to generate
artificial neural networks, parameters, topology and
rules. It is most commonly applied in artificial life,
general game playing and evolutionary robotics.”
— Wikipedia
“The evolution of neural networks is called neuroevolution”
— Hannah Le

31. 31
Idea of Neuroevolution
Supervised learning algorithms require correct
input-output pairs
Neuroevolution only requires a measure of a
network’s performance
A major inspiration for the investigation of
neuroevolution is the evolution of brains in nature

32. 32
Examples of neuroevolution

33. 33
MarIO
https://www.youtube.com/watch?v=qv6UVOQ0F44

34. 34
Flappy Bird
https://github.com/xviniette/FlappyLearning

35. 35
A simple algorithm for neuroevolution

36. 36
Simple Algorithm
Use Genetic Algorithm to search for the weights
and bias of each neuron
The fitness function to optimize could be based
on the error function (e.g., MSE loss)

37. 37

38. 38

39. 39

40. 40

41. 41

42. 42

43. 43

44. 44
New neural network
With a better accuracy

45. 45
Genetic operations
Applying genetic algorithm to find weights and bias of
a neural network involves particular genetic operations
The cross-over operations must consider the neuron
as a whole
ie. Crossover should not divide the genetic information obtained
from a neuron
Cross-over happens only at a limit of a neuron
Because of the competing convention we will later see

46. 46
Variant of crossover
We have originally presented GA with two genetic
operations
In fact, we can run GA with more operations
You can also have an additional second cross-
over operations that crosscut only at the limit of a
layer
Cross-over happens at the junction of a layer

47. 47
Genetically modified neural network
Neuroevolution is convenient when training data is
either missing, insufficient, or unusable (eg bad
quality, imbalanced)
The small algorithm we have just seen is effective
in many situations
Iris or seed data set
AI for a simple game (e.g., Flappy bird)

48. 48
What about network architecture?
This simple algorithm finds weights and bias while
maintaining a fixed network structure
Structure of the network and some of its
parameters can also be discovered
The idea of searching for a network architecture is
to look for simple and minimal networks
Actually, fully-connected forward networks are
pretty far away from what nature has brought us

49. 49
NEAT — NeuroEvolution of Augmenting Topologies
Kenneth O. Stanley and Risto Miikkulainen. 2002.
Evolving neural networks through augmenting
topologies
Evolutionary Computation archive
Volume 10 Issue 2, Summer 2002
Pages 99-127
http://dx.doi.org/10.1162/106365602320169811

50. 50
Essence of NEAT
Genetic algorithm that produces neural networks
NEAT varies the structure of the network, producing
complex solutions over time
NEAT needs its overall population of networks to be diverse
The reproduction phase should produces a novel individual
NEAT groups similar individuals into species
Direct competition for survival happens only within a
species

51. 51
Essence of NEAT
Species compete with each other for
reproduction rights
Mating only happens within species (unless
interspecies matin is explicitly allowed)
Because NEAT uses species, it takes longer for
the whole population to become uniform

52. 52
Limitation of gradient-based approaches
Neuroevolution can learn things that classical
neural networks cannot learn
activation functions, hyperparameter, architectures

53. 53
Encoding

54. 54
Encoding
Inputs and outputs nodes are not encoded
Hidden nodes can be added or removed
A connection may be enabled or not, and has an innovation number
Individuals do not necessarily have the same size

55. 55
Mutation

56. 56
Mutation
If a new node is added:
1 - it is placed between two connected nodes
2 - the previous connection is disabled (but still present in the
genome)

57. 57
Competing Convention Problem
In ordinary evolution algorithms it can happen that
two individuals encode very similar behavior, but
with very different genotype
This is called competing conventions
If these individual are subject to crossover, their
children are likely to be worse than their either
parent

58. 58
Competing Convention Problem

59. 59
Competing Convention Problem
NEAT solves this by keeping historical markings of
new structural elements
When a new structural element is created (via
structural mutation, such as adding a new node or
gene), it receives an innovation number
When two individuals are crossed over, their
genotypes are aligned in such a way that the
corresponding innovation numbers match and only the
different elements are exchanged

60. 60

61. 61
Speciation
Most new evolutions are not good ones
A new connection or node degrades the
performance of the individual
How can we protect new networks from being
eliminated?
NEAT suggests speciation

62. 62
Speciation
Splits up the population into several species
Based on the similarity of topology and connections
Individuals in a population only have to compete
within other individuals within that species
Allow for a new network to be created and optimized
without being quickly eliminated
Explore the idea of explicit fitness sharing: the whole
population has a fitness

63. 63
Speciation
Speciation project innovations in the population
When a network diverges far enough from other
networks in the population, NEAT identifies it and
put it in its own species
Using innovation numbers NEAT computes the
distance between two genomes

64. 64
Minimal structure
NEAT favors minimal networks to be evolved
Mutations is about adding a node, adding a
connection, or changing a weight
… and not removing structural elements

65. 65
What have we seen?
An overview of Genetic Algorithm
Two algorithms for Neuroevolution
Simple: Blindly applying GA to Deep learning
NEAT: Use of species to achieve incremental
evolution of network complexity

66. 66
Extensions of NEAT
Many improvements were proposed around NEAT
HyperNEAT: indirect encoding of network
ESHyperNEAT: higher density of neurons when
necessary
CoDeepNEAT: macro systems
Indirect coding using rules mimicking cell divisions

67. 67
References
Designing neural networks through neuroevolution —
Nature Machine Intelligence volume 1, 24–35(2019)
A Hypercube-Based Indirect Encoding for Evolving
Large-Scale Neural Networks — Artificial Life journal
15(2), Cambridge, MA: MIT Press, 2009