SlideShare a Scribd company logo
1 of 56
Download to read offline
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
.
.
. ..
.
.
Robustness and evolution of novel functions
in programmable hardware
Karthik Raman
Andreas Wagner Lab
Institute of Evolutionary Biology and Environmental Studies
University of Zürich
April 1, 2011
1 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Outline
.
. .1 Introduction
Robustness and Evolvability
Neutral Networks
.
. .2 Genotype Space and Neutral Networks
Genotype space
Phenotype space
Neutral networks
.
. .3 Robustness and Evolvability
Robustness of circuits
Evolving new functions
.
. .4 Summary
2 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Biological systems vs. Man-made systems
.
Biological systems
..
.
. ..
.
.
Shaped through the forces of mutation and natural selection
Survive onslaught of disruptive agents — hostile
environments/random mutations: robust
Show a remarkable ability to adapt and evolve novel properties
through such random mutations: evolvable
.
Man-made systems
..
.
. ..
.
.
Product of rational design, rather than biological evolution
Generally not as robust: often fragile — modification/removal of
components results in catastrophic failure
Their ability to acquire novel and useful features through random
change is limited
3 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Biological systems vs. Man-made systems
.
Biological systems
..
.
. ..
.
.
Shaped through the forces of mutation and natural selection
Survive onslaught of disruptive agents — hostile
environments/random mutations: robust
Show a remarkable ability to adapt and evolve novel properties
through such random mutations: evolvable
.
Man-made systems
..
.
. ..
.
.
Product of rational design, rather than biological evolution
Generally not as robust: often fragile — modification/removal of
components results in catastrophic failure
Their ability to acquire novel and useful features through random
change is limited
3 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Biological systems vs. Man-made systems
.
Biological systems
..
.
. ..
.
.
Shaped through the forces of mutation and natural selection
Survive onslaught of disruptive agents — hostile
environments/random mutations: robust
Show a remarkable ability to adapt and evolve novel properties
through such random mutations: evolvable
.
Man-made systems
..
.
. ..
.
.
Product of rational design, rather than biological evolution
Generally not as robust: often fragile — modification/removal of
components results in catastrophic failure
Their ability to acquire novel and useful features through random
change is limited
3 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Robustness
Fundamental feature of most biological systems — ability to continue
normal function in the face of perturbations
Man-made systems — generally not very robust to failure
Design of fault-tolerant systems is receiving increased attention
4 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Evolvability
“The ability to produce phenotypic diversity, novel solutions to the problems
faced by organisms and evolutionary innovations”
Common feature of biological systems — novel functions, which help an
organism survive and reproduce, are acquired through genetic change.
The need for adaptation in artificial systems is being recognised — ability
to learn and adapt to new situations, e. g. autonomous robots
5 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Evolvable Hardware
Principles of evolution have been applied in computer science for
solving many complex optimisation problems — evolutionary
computationa
implements aspects such as random variation, reproduction,
selection, . . .
Artificial evolution can automatically generate designs of digital
circuits, as well as circuits that are robust to noise and faultsb
Evolvable hardware = evolutionary algorithms applied to electronic
circuits and devices
a
Foster JA (2001) Nat Rev Genet 2:428–436
b
Hartmann M & Haddow P (2004) IEE Proceedings - Computers and Digital Techniques 151:287–294
6 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Key Questions
How ‘robust’ is a typical circuit to changes in the
wiring/configuration?
Do neutral networks exist in this circuit space?
Can circuits with significantly different configuration compute the
same Boolean function?
Does the organisation of neutral networks facilitate the adoption of
novel phenotypes (logic function computations) through small
numbers of gate changes?
7 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
Neutral networks — Genotypes sharing the same phenotype
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
Neutral networks — Genotypes sharing the same phenotype
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
Neutral networks — Genotypes sharing the same phenotype
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
Neutral networks — Genotypes sharing the same phenotype
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
Neutral networks — Genotypes sharing the same phenotype
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks
Genotype space
Nodes: Genotypes
Edges: mutation/simple genetic
change
Phenotypes mapped to genotypes
Largest neutral network
Smaller neutral network
Smaller and more fragmented neutral
set
Disconnected multiple neutral sets
Neutral networks — Genotypes sharing the same phenotype
The genotype space is covered with multiple neutral sets/networks
Neutral networks have implications for robustness/evolvability
8 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Genotype vs. Phenotype
Genotype evolves through mutation and is inherited; selection acts
on phenotype
Genotype–phenotype mapping — required to distinguish neutral
networks
Mapping may not be straightforward always
System Genotype Phenotype
RNA/Proteins Sequence Structure
Regulatory networks Regulatory interactions Gene expression patterns
Circadian oscillators Regulatory interactions Oscillations
Digital circuits Circuit components/wiring Boolean function computed
Protein–protein interactions Network topology ?
Signal transduction Network topology ?
9 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral networks vs. Robustness/Evolvability
Neutral networks can be traversed in small evolutionary steps,
through simple changes/mutations
Larger neutral network ⇒ higher robustness, scope for evolutionary
innovation …
10 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks vs. Robustness
Time
High robustness
Low robustness
Low robustness: more deleterious mutations
High robustness: likely to encounter more novel phenotypes 11 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks vs. Robustness
Time
High robustness
Low robustness
Low robustness: more deleterious mutations
High robustness: likely to encounter more novel phenotypes 11 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral Networks vs. Robustness
Time
High robustness
Low robustness
—Wagner A (2008a) Nat Rev Genet 9:965–974
Low robustness: more deleterious mutations
High robustness: likely to encounter more novel phenotypes 11 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Robustness vs. Evolvability
Robustness and evolvability — both correlate with neutral network
size/connectivity
Robust phenotypes tend to have higher evolvability
Low Robustness
Low Innovation
High Robustness
Low Innovation
High Robustness
High Innovation
—Ciliberti S et al. (2007) Proc Natl Acad Sci U S A 104:13591–13596
Populations evolving on larger neutral networks have greater access
to variation*
12 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Outline
.
. .1 Introduction
Robustness and Evolvability
Neutral Networks
.
. .2 Genotype Space and Neutral Networks
Genotype space
Phenotype space
Neutral networks
.
. .3 Robustness and Evolvability
Robustness of circuits
Evolving new functions
.
. .4 Summary
13 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Field Programmable Gate Arrays (FPGAs)
I
N
P
U
T
S
1
2
3
4
6
7
O
U
T
P
U
T
S
5 8
9
10 13
12
11
OR
A
B
Y
AND
A
B
Y
NAND
A
B
Y
NOR
A
B
Y
XOR
A
B
Y
Y=A+B
Y=A×B
Y=AÅB
___
Y=(A×B)
____
Y=(A+B)
n × m array of logic gates
Configuration can be dynamically reprogrammed: ‘field
programmable’
Maps a Boolean input of nI variables to an output of no variables:
f : {0, 1}nI
→ {0, 1}no
14 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Generic FPGA
inputs
outputs
nI inputs, mapped to no outputs
All nI inputs must be used in column 1
n × m array of logic gates, Lij, each with two inputs
Inter-connections are feed-forward: a gate in column i can take an
input from any of the previous levels
no outputs can come from any of the mn gate outputs
15 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Representation of FPGAs
I
N
P
U
T
S
1
2
3
4
6
7
O
U
T
P
U
T
S
5 8
9
10 13
12
11
4 2 3
L11
1 1 5
L12
3 2 2
L13
2 2 5
L21
1 7 2
L22
5 2 3
L23
10 5 3
L31
3 3 1
L32
6 6 4
L33
11 12 10 13
Outputs
Size of representation = 3mn + no = diameter of circuit space
[Similar to Cartesian Genetic Programming — Miller & Thomson (2000)]
16 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Genotype space: Circuits with various configurations
I
N
P
U
T
S
1
2
4
53
6
O
U
T
P
U
T
S
I
N
P
U
T
S
1
2
4
53
6
O
U
T
P
U
T
S
I
N
P
U
T
S
1
2
4
53
6
O
U
T
P
U
T
S
I
N
P
U
T
S
1
2
4
53
6
O
U
T
P
U
T
S
I
N
P
U
T
S
1
2
4
53
6
O
U
T
P
U
T
S
C1 C2
C3C4C5
C6
I
N
P
U
T
S
1
2
4
53
6
O
U
T
P
U
T
S
Every node in genotype space is a circuit (of a particular size)
Neighbours vary by a small elementary change in: (a) Internal wiring
(C2), (b) Logic function of one of the gates (C3), (c) Input mapping
(C5), or (d) Output mapping (C6)
d(C4, C1) = 1 ⇒ not neighbours
17 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Phenotype Space: Boolean functions
Every circuit maps a Boolean input of nI bits to an output of no bits:
f : {0, 1}nI
→ {0, 1}no
For example,
i j k l ⇒ j k l i
Circular [Left] Shift
i j k l ⇒ 0 i j k
[Unsigned] Right Shift
I1 I2 I3 I4 O1 O2 O3 O4
0 0 0 0 0 0 0 0
0 0 0 1 0 0 1 0
0 0 1 0 0 1 0 0
0 0 1 1 0 1 1 0
0 1 0 0 1 0 0 0
0 1 0 1 1 0 1 0
0 1 1 0 1 1 0 0
0 1 1 1 1 1 1 0
1 0 0 0 0 0 0 1
1 0 0 1 0 0 1 1
1 0 1 0 0 1 0 1
1 0 1 1 0 1 1 1
1 1 0 0 1 0 0 1
1 1 0 1 1 0 1 1
1 1 1 0 1 1 0 1
1 1 1 1 1 1 1 1
Circular left shift
truth table
Total Boolean functions = 2entries in truth table
= 264
= 1.84 × 1019
18 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Sampling circuit space
Sample size = 2 × 107
Circuit space sampled uniformly
FPGA size Genotype
space size
Functions observed
in sample
3 × 3 1.08 ×1024
1.59 ×107
4 × 4 8.43 ×1045
1.74 ×107
5 × 5 2.90 ×1076
1.80 ×107
6 × 6 6.05 ×10116
1.83 ×107
Total Boolean functions = 264
= 1.84 × 1019
Small fraction of total Boolean functions observed: ≈ 10−12
Most observed functions are computed only by one circuit in the
sample
Few functions are computed by large number of circuits
19 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Majority of functions are rare
10
0
10
5
10
−8
10
−7
10
−6
10
−5
10
−4
Rank of logic function (decreasing size of circuit sets)
Frequencyoflogicfunction
Some functions appear over a 1000
times
Long tail: Huge number of functions
appear only once
Functions such as right shift/circular
shift do not appear in the sample of
2 × 107
20 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neutral networks in circuit space
Circuits computing the same function can be reached through
function-preserving random walks in circuit space
⇒ Circuit space is spanned by large neutral networks
0 0.2 0.4 0.6 0.8 1
10
0
10
1
10
2
10
3
Maximal circuit distance (D)
Numberoffunctions
4x4 Circuits
10
−7
10
−6
10
−5
10
−4
0.3
0.5
0.7
1
Frequency of the logic functionMaximalcircuitdistance(D)
4x4 Circuits
Spearman’s r = 0.56; p < 10−300
; n = 1000
A function-preserving random walk in circuit space can take us very
far away from the initial circuit
⇒ Neutral networks have very large diameter
21 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Diverse circuits can compute the same function
I
N
P
U
T
S
1
2
3
4
6
7
O
U
T
P
U
T
S
5 8
9
10 13
12
11
4 2 3 1 1 5 3 2 2 2 2 5 1 7 2 5 2 3 10 5 3 3 3 1 6 6 4 11 12 10 13
I
N
P
U
T
S
1
2
3
4
6
7
O
U
T
P
U
T
S
5 8
9
10 13
12
11
2 3 1 4 4 1 1 1 3 5 3 4 7 6 1 7 3 1 1 1 2 1 5 2 5 2 2 13 10 9 11
Both circuits compute
circular shift
Differ in every logic
gate, wiring, input &
output mapping
Thus, the diameters of
neutral networks can be
quite large
22 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Outline
.
. .1 Introduction
Robustness and Evolvability
Neutral Networks
.
. .2 Genotype Space and Neutral Networks
Genotype space
Phenotype space
Neutral networks
.
. .3 Robustness and Evolvability
Robustness of circuits
Evolving new functions
.
. .4 Summary
23 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Robustness to change in configuration
10
−7
10
−6
10
−5
10
−4
0
0.2
0.4
0.6
0.8
1
Frequency of the logic function
Robustnesstoconfigurationchange
4x4 Circuits
Spearman’s r = 0.32; p < 10
−300
0 0.2 0.4 0.6 0.8 1
0
50
100
Circular shift (3x3)
0 0.2 0.4 0.6 0.8 1
0
20
40
Numberofcircuits
Circular shift (4x4)
0 0.2 0.4 0.6 0.8 1
0
20
40
Fraction of neutral neighbors
Circular shift (6x6)
Robustness = fraction of neutral neighbours
Robustness distribution is quite broad
Larger circuits are more robust (regardless of function)
24 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Robustness to gate failure
10
−7
10
−6
10
−5
10
−4
0
0.2
0.4
0.6
0.8
1
Frequency of the logic function
Robustnesstogatefailure
4x4 Circuits
0 0.2 0.4 0.6 0.8 1
0
50
Circular shift (3x3)
0 0.2 0.4 0.6 0.8 1
0
20
40
Numberofcircuits
Circular shift (4x4)
0 0.2 0.4 0.6 0.8 1
0
20
40
Failure robustness
Circular shift (6x6)
Gate failures are common in digital circuits
Types of failures considered: zero output (and inverted output)
Failure robustness = fraction of single gate failures that do not affect
the computed function
Failure robustness distribution is also quite broad
Larger circuits are more robust to gate failure
25 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Evolving new functions
On large neutral networks, genotypes can change substantially
without changing phenotype
Phenotypes in different neighbourhoods of a neutral network can be
quite different
In biological systems, this feature facilitates the exploration of new
phenotypesab
For circuits, this will have implications for the ease with which
evolvable hardware can acquire new functions
a
Ciliberti S et al. (2007) Proc Natl Acad Sci U S A 104:13591–13596
b
Wagner A (2008b) Proc Biol Sci 275:91–100
26 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neighbourhood of evolving circuits
0 500 1000 1500 2000
0
500
1000
1500
2000
2500
3000
Random walk steps
Cumulativenovelfunctions
encounteredinneighborhood
0 1000 2000
0
1000
2000
3000
f = 5.00 ⋅ 10
−8
Cumulativenovelfunctions
encounteredinneighborhood
0 1000 2000
0
1000
2000
3000
f = 5.00 ⋅ 10
−8
0 1000 2000
0
1000
2000
3000
f = 3.00 ⋅ 10
−7
0 1000 2000
0
1000
2000
3000
f = 4.65 ⋅ 10
−6
0 1000 2000
0
1000
2000
3000
f = 6.35 ⋅ 10−6
Random walk steps
Cumulativenovelfunctions
encounteredinneighborhood
0 1000 2000
0
1000
2000
3000
f = 9.40 ⋅ 10−6
Random walk steps
0 1000 2000
0
1000
2000
3000
f = 1.24 ⋅ 10−5
Random walk steps
0 1000 2000
0
1000
2000
3000
f = 5.09 ⋅ 10−5
Random walk steps
As we explore the neutral network (through a function-preserving
random walk), a large number of new phenotypes are encountered
The number of new phenotypes keeps increasing, as we prolong the
random walk
This property holds for a large number of functions
Indicates the high accessibility of innovation in the neighbourhood
27 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neighbourhood of evolving circuits
What is the fraction of new phenotypes encountered in the
neighbourhood of any circuit?
With reference to a starting circuit C0,
u(C0, Ci) = 1 −
|N0 ∩ Ni|
|N0 ∪ Ni|
28 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neighbourhood of evolving circuits
What is the fraction of new phenotypes encountered in the
neighbourhood of any circuit?
With reference to a starting circuit C0,
u(C0, Ci) = 1 −
|N0 ∩ Ni|
|N0 ∪ Ni|
G1
G2
shared
unique
u(G1, G2) = = 5
8
= 0.625
28 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Neighbourhood of evolving circuits
0 50 100 150 200
0
0.2
0.4
0.6
0.8
1
Random walk steps
Fractionofnewfunctions
inneighborhood
0 1000 2000
0
0.5
1
f = 5.00 ⋅ 10
−8
Fractionofnewfunctions
inneighborhood
0 1000 2000
0
0.5
1
f = 5.00 ⋅ 10
−8
0 1000 2000
0
0.5
1
f = 3.00 ⋅ 10
−7
0 1000 2000
0
0.5
1
f = 4.65 ⋅ 10
−6
0 1000 2000
0
0.5
1
f = 6.35 ⋅ 10−6
Random walk steps
Fractionofnewfunctions
inneighborhood
0 1000 2000
0
0.5
1
f = 9.40 ⋅ 10−6
Random walk steps
0 1000 2000
0
0.5
1
f = 1.24 ⋅ 10−5
Random walk steps
0 1000 2000
0
0.5
1
f = 5.09 ⋅ 10−5
Random walk steps
Beyond the distance of ≈ one network diameter, over 80% of the
functions found in the neighbourhood are new
At increasing distances from a circuit, a very large number of new
phenotypes are accessible; this property holds for a large number of
functions
29 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Fraction of new phenotypes at the end of a random walk
10
−7
10
−6
10
−5
10
−4
0
0.2
0.4
0.6
0.8
1
Frequency of the logic function
Fractionofnewfunctionsin
neighborhoodatendofrandomwalk
4x4 Circuits
0 0.2 0.4 0.6 0.8 1
0
10
20
30
40
50
60
Fraction of new functions in neighborhood
at the end of random walk
Numberofcircuits
Circular shift (4x4)
A lot of innovation is accessible in the neighbourhood, once we walk
sufficiently far awaya
from a starting circuit . . .
a
2,000 steps, in this case
30 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Proximity of neutral networks
How far must one travel in circuit space from one neutral set, to find
another neutral set?
31 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Proximity of neutral networks
How far must one travel in circuit space from one neutral set, to find
another neutral set?
0 0.2 0.4 0.6 0.8 1
0
50
100
150
200
Minimum distance between circuits
computing right shift and circular shift
(as fraction of circuit space diameter)
Pairsofcircuits
0 0.2 0.4 0.6 0.8 1
0
200
400
600
800
Minimum distance between neutral networks
(as fraction of circuit space diameter)
Pairsofneutralnetworksd(CS, RS) ≤ 3!
It is possible to discover new functions through a relatively small
number of elementary changes
31 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Similarity with Biological Systems
The system of circuits studied show intriguing similarities with biological
systems:
Existence of large highly connected neutral networks
Large number of diverse genotypes adopt the same phenotype
Robustness to change in configurations
Large number of novel phenotypes accessible in the neighbourhood
of evolving genotypes
Proximity of different neutral networks
Importance of “junk” non-functional components …
32 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Role of ‘non-functional components’
Many circuits have non-functional gates, system parts that are not involved in the
computation a given circuit carries out
In biological systems, many amino acids in a proteinab
, many regulatory interactions
in a gene regulation circuitc
, and many metabolic reactions in a metabolic reaction
networkd
may appear as ‘non-functional’ or ‘dispensable’
Tempting to call such system parts ‘junk’ parts — however, such parts play a crucial
role for evolvability, and it is precisely their ability to vary freely in some
environments that allows biological systems to evolve novel phenotypes
For example, in laboratory evolution experiments, proteins with new function evolve
often through changes that do not affect the protein’s principal functionb
Circuits of a minimal size may have the merit of computing a function in an elegant
and simple way — at the same time, they would be utterly unevolvable
Evolvability comes at the price of high complexity
a
Bloom JD et al. (2005) Proc Natl Acad Sci U S A 102:606–611
b
Aharoni A et al. (2005) Nat Genet 37:73–76
c
Ciliberti S et al. (2007) Proc Natl Acad Sci U S A 104:13591–13596
d
Matias Rodrigues JF & Wagner A (2009) PLoS Comput Biol 5:e1000613
33 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Outline
.
. .1 Introduction
Robustness and Evolvability
Neutral Networks
.
. .2 Genotype Space and Neutral Networks
Genotype space
Phenotype space
Neutral networks
.
. .3 Robustness and Evolvability
Robustness of circuits
Evolving new functions
.
. .4 Summary
34 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Limitations
Many other factors decide the suitability of a particular circuit, e. g.
power consumption, robustness to temperature variations, and
trade-offs between functional flexibility and performance
Fine differences exist between two silicon chips – circuits evolved on
one silicon chip are not guaranteed to work on anothera
!
Differences exist, between software simulation and hardware
evolution
Some properties of our (or any other) study system may depend on
the choice of representation for a circuit’s architecture
Possible major computational issues with very large circuit sizes
Due to the astronomical numbers of circuits and functions, one
needs to resort to sampling to understand circuit space — not
limiting if one is interested in generic properties of this space
a
Thompson A (1995) In: F Morán, A Moreno, JJ Merelo, & P Chacon (eds.), Advances in Artificial Life:
Proc. 3rd Eur. Conf. on Artificial Life (ECAL95), volume 929 of LNAI, pages 640–656. Springer-Verlag
35 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Summary
A random sampling of circuit space reveals that majority of the
functions are computed by very few circuits
Circuits computing the same function form large connected neutral
networks with large diameters
Diverse circuits can compute the same function
Larger circuits tend to be more robust; the distribution of robustness
is broad
for a given function and size, the most robust configuration may be
useful in applications
A number of new functions are accessible in the neighbourhood of
evolving circuits
evolvable circuits may find application in the design of adaptive
devices
Neutral networks are also located close together in space
access to new functions through few changes
‘minimal reconfiguration’ may be particularly useful/efficient
36 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Summary
A random sampling of circuit space reveals that majority of the
functions are computed by very few circuits
Circuits computing the same function form large connected neutral
networks with large diameters
Diverse circuits can compute the same function
Larger circuits tend to be more robust; the distribution of robustness
is broad
for a given function and size, the most robust configuration may be
useful in applications
A number of new functions are accessible in the neighbourhood of
evolving circuits
evolvable circuits may find application in the design of adaptive
devices
Neutral networks are also located close together in space
access to new functions through few changes
‘minimal reconfiguration’ may be particularly useful/efficient
36 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Summary
A random sampling of circuit space reveals that majority of the
functions are computed by very few circuits
Circuits computing the same function form large connected neutral
networks with large diameters
Diverse circuits can compute the same function
Larger circuits tend to be more robust; the distribution of robustness
is broad
for a given function and size, the most robust configuration may be
useful in applications
A number of new functions are accessible in the neighbourhood of
evolving circuits
evolvable circuits may find application in the design of adaptive
devices
Neutral networks are also located close together in space
access to new functions through few changes
‘minimal reconfiguration’ may be particularly useful/efficient
36 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Summary
A random sampling of circuit space reveals that majority of the
functions are computed by very few circuits
Circuits computing the same function form large connected neutral
networks with large diameters
Diverse circuits can compute the same function
Larger circuits tend to be more robust; the distribution of robustness
is broad
for a given function and size, the most robust configuration may be
useful in applications
A number of new functions are accessible in the neighbourhood of
evolving circuits
evolvable circuits may find application in the design of adaptive
devices
Neutral networks are also located close together in space
access to new functions through few changes
‘minimal reconfiguration’ may be particularly useful/efficient
36 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Summary
A random sampling of circuit space reveals that majority of the
functions are computed by very few circuits
Circuits computing the same function form large connected neutral
networks with large diameters
Diverse circuits can compute the same function
Larger circuits tend to be more robust; the distribution of robustness
is broad
for a given function and size, the most robust configuration may be
useful in applications
A number of new functions are accessible in the neighbourhood of
evolving circuits
evolvable circuits may find application in the design of adaptive
devices
Neutral networks are also located close together in space
access to new functions through few changes
‘minimal reconfiguration’ may be particularly useful/efficient
36 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Outlook: Exploiting robustness and evolvability
.
In technological systems
..
.
. ..
.
.
For any desired Boolean function, it may be beneficial to use the
most robust circuit (one that is resistant to a number of gate failures
as well as configuration changes) of a particular size
Circuits that have access to a greater amount of innovation (diverse
Boolean functions), may be useful in designing systems with adaptive
behaviour
.
In biology …
..
.
. ..
.
.
Such circuits can also be viewed as generalised models of signalling
networks/gene regulatory networks
Such circuits also find applications in synthetic biology, for the
design of synthetic circuitsa
a
Marchisio MA & Stelling J (2011) PLoS Comput Biol 7:e1001083+
37 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Not all circuits computing the same function are equal …
I
N
P
U
T
S
1
2
3
4
6
7
O
U
T
P
U
T
S
5
8
Circuit A
Robustness = 0.0556
I
N
P
U
T
S
1
2
3
4
O
U
T
P
U
T
S
1915117
2016128
1814106
171395
Circuit B
Robustness = 0.5725
…is complexity the price for evolvability?
38 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
Thank you
39 / 40
. . . . . . . . . .
Introduction
. . . . . . . . .
Genotype Space and Neutral Networks
. . . . . . . . . .
Robustness and Evolvability
. . . . . .
Summary
40 / 40

More Related Content

Viewers also liked

Sir Francis Bacon
Sir Francis BaconSir Francis Bacon
Sir Francis BaconNaj_Jandy
 
Of marrige and single life (euro. lit.)
Of marrige and single life (euro. lit.)Of marrige and single life (euro. lit.)
Of marrige and single life (euro. lit.)Ysa Garcera
 
Henry Fielding as a Novelist
Henry Fielding as a NovelistHenry Fielding as a Novelist
Henry Fielding as a NovelistBharat008
 
Mikhail mikhailovich bakhtin, lecture outline
Mikhail mikhailovich bakhtin, lecture outlineMikhail mikhailovich bakhtin, lecture outline
Mikhail mikhailovich bakhtin, lecture outlineHina Honey
 
Novels , society & history
Novels , society & historyNovels , society & history
Novels , society & historyamyhs_97
 
History of novels
History of novelsHistory of novels
History of novelsnn555818
 
Literary Genres
Literary GenresLiterary Genres
Literary Genreskrporter
 
Timeline and Characteristics of British Literature
Timeline and Characteristics of British LiteratureTimeline and Characteristics of British Literature
Timeline and Characteristics of British LiteratureJavier Aguirre
 
English Literature
English LiteratureEnglish Literature
English LiteratureTarık İnce
 

Viewers also liked (20)

Joseph Andrews
Joseph Andrews Joseph Andrews
Joseph Andrews
 
Sir Francis Bacon
Sir Francis BaconSir Francis Bacon
Sir Francis Bacon
 
Of marrige and single life (euro. lit.)
Of marrige and single life (euro. lit.)Of marrige and single life (euro. lit.)
Of marrige and single life (euro. lit.)
 
Unsur intrinsik novel
Unsur intrinsik novelUnsur intrinsik novel
Unsur intrinsik novel
 
Henry Fielding as a Novelist
Henry Fielding as a NovelistHenry Fielding as a Novelist
Henry Fielding as a Novelist
 
Joseph andrews
Joseph andrewsJoseph andrews
Joseph andrews
 
Mikhail mikhailovich bakhtin, lecture outline
Mikhail mikhailovich bakhtin, lecture outlineMikhail mikhailovich bakhtin, lecture outline
Mikhail mikhailovich bakhtin, lecture outline
 
What is a novel: history and observations
What is a novel: history and observationsWhat is a novel: history and observations
What is a novel: history and observations
 
Novels , society & history
Novels , society & historyNovels , society & history
Novels , society & history
 
Francis Bacon
Francis BaconFrancis Bacon
Francis Bacon
 
History of novels
History of novelsHistory of novels
History of novels
 
Literary genres
Literary genres Literary genres
Literary genres
 
Literary Genres
Literary GenresLiterary Genres
Literary Genres
 
Timeline and Characteristics of British Literature
Timeline and Characteristics of British LiteratureTimeline and Characteristics of British Literature
Timeline and Characteristics of British Literature
 
Novel
NovelNovel
Novel
 
Types of Genres
Types of GenresTypes of Genres
Types of Genres
 
Fiction genres
Fiction genresFiction genres
Fiction genres
 
Ages of English Literature
Ages of English LiteratureAges of English Literature
Ages of English Literature
 
Genre Analysis
Genre AnalysisGenre Analysis
Genre Analysis
 
English Literature
English LiteratureEnglish Literature
English Literature
 

Similar to Robustness and evolution of novel functions in programmable hardware

Manuscrit de Doctorat_El Abdellaouy Hanane
Manuscrit de Doctorat_El Abdellaouy HananeManuscrit de Doctorat_El Abdellaouy Hanane
Manuscrit de Doctorat_El Abdellaouy HananeElabdellaouy Hanane
 
file_format_presentation
file_format_presentationfile_format_presentation
file_format_presentationRussell Jarvis
 
Analysis and Simulation of Scienti c Networks
Analysis and Simulation of Scientic NetworksAnalysis and Simulation of Scientic Networks
Analysis and Simulation of Scienti c NetworksFelix Puetsch
 
MIMO-OFDM communication systems_ channel estimation and wireless.pdf
MIMO-OFDM communication systems_  channel estimation and wireless.pdfMIMO-OFDM communication systems_  channel estimation and wireless.pdf
MIMO-OFDM communication systems_ channel estimation and wireless.pdfSamerSamerM
 
Effect of antenna configurations
Effect of antenna configurationsEffect of antenna configurations
Effect of antenna configurationsmarwaeng
 
Emona-based Interactive Amplitude Modulation/Demodulation iLab
Emona-based Interactive Amplitude Modulation/Demodulation iLabEmona-based Interactive Amplitude Modulation/Demodulation iLab
Emona-based Interactive Amplitude Modulation/Demodulation iLabHuynh MVT
 
Dissertation wonchae kim
Dissertation wonchae kimDissertation wonchae kim
Dissertation wonchae kimSudheer Babu
 
SeniorThesisFinal_Biswas
SeniorThesisFinal_BiswasSeniorThesisFinal_Biswas
SeniorThesisFinal_BiswasAditya Biswas
 
Extended LTE Coverage for Indoor Machine Type Communication.pdf
Extended LTE Coverage for Indoor Machine Type Communication.pdfExtended LTE Coverage for Indoor Machine Type Communication.pdf
Extended LTE Coverage for Indoor Machine Type Communication.pdfYAAKOVSOLOMON1
 
Nonlinear image processing using artificial neural
Nonlinear image processing using artificial neuralNonlinear image processing using artificial neural
Nonlinear image processing using artificial neuralHưng Đặng
 
pawel_jakubowski_master_thesis_swarm
pawel_jakubowski_master_thesis_swarmpawel_jakubowski_master_thesis_swarm
pawel_jakubowski_master_thesis_swarmPaweł Jakubowski
 
02 whole
02 whole02 whole
02 wholevino205
 
OFDM Based Cognitive radio
OFDM Based Cognitive radioOFDM Based Cognitive radio
OFDM Based Cognitive radioDeepesh Sinnya
 

Similar to Robustness and evolution of novel functions in programmable hardware (20)

Manuscrit de Doctorat_El Abdellaouy Hanane
Manuscrit de Doctorat_El Abdellaouy HananeManuscrit de Doctorat_El Abdellaouy Hanane
Manuscrit de Doctorat_El Abdellaouy Hanane
 
file_format_presentation
file_format_presentationfile_format_presentation
file_format_presentation
 
Analysis and Simulation of Scienti c Networks
Analysis and Simulation of Scientic NetworksAnalysis and Simulation of Scientic Networks
Analysis and Simulation of Scienti c Networks
 
978-3-659-82929-1
978-3-659-82929-1978-3-659-82929-1
978-3-659-82929-1
 
MIMO-OFDM communication systems_ channel estimation and wireless.pdf
MIMO-OFDM communication systems_  channel estimation and wireless.pdfMIMO-OFDM communication systems_  channel estimation and wireless.pdf
MIMO-OFDM communication systems_ channel estimation and wireless.pdf
 
Effect of antenna configurations
Effect of antenna configurationsEffect of antenna configurations
Effect of antenna configurations
 
edc_adaptivity
edc_adaptivityedc_adaptivity
edc_adaptivity
 
Thesis
ThesisThesis
Thesis
 
thesis
thesisthesis
thesis
 
Emona-based Interactive Amplitude Modulation/Demodulation iLab
Emona-based Interactive Amplitude Modulation/Demodulation iLabEmona-based Interactive Amplitude Modulation/Demodulation iLab
Emona-based Interactive Amplitude Modulation/Demodulation iLab
 
Dissertation wonchae kim
Dissertation wonchae kimDissertation wonchae kim
Dissertation wonchae kim
 
SeniorThesisFinal_Biswas
SeniorThesisFinal_BiswasSeniorThesisFinal_Biswas
SeniorThesisFinal_Biswas
 
Extended LTE Coverage for Indoor Machine Type Communication.pdf
Extended LTE Coverage for Indoor Machine Type Communication.pdfExtended LTE Coverage for Indoor Machine Type Communication.pdf
Extended LTE Coverage for Indoor Machine Type Communication.pdf
 
Nonlinear image processing using artificial neural
Nonlinear image processing using artificial neuralNonlinear image processing using artificial neural
Nonlinear image processing using artificial neural
 
Tac note
Tac noteTac note
Tac note
 
Final Thesis Report
Final Thesis ReportFinal Thesis Report
Final Thesis Report
 
pawel_jakubowski_master_thesis_swarm
pawel_jakubowski_master_thesis_swarmpawel_jakubowski_master_thesis_swarm
pawel_jakubowski_master_thesis_swarm
 
Stocchi tesi
Stocchi tesiStocchi tesi
Stocchi tesi
 
02 whole
02 whole02 whole
02 whole
 
OFDM Based Cognitive radio
OFDM Based Cognitive radioOFDM Based Cognitive radio
OFDM Based Cognitive radio
 

Recently uploaded

Natural Polymer Based Nanomaterials
Natural Polymer Based NanomaterialsNatural Polymer Based Nanomaterials
Natural Polymer Based NanomaterialsAArockiyaNisha
 
Artificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C PArtificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C PPRINCE C P
 
Boyles law module in the grade 10 science
Boyles law module in the grade 10 scienceBoyles law module in the grade 10 science
Boyles law module in the grade 10 sciencefloriejanemacaya1
 
Grafana in space: Monitoring Japan's SLIM moon lander in real time
Grafana in space: Monitoring Japan's SLIM moon lander  in real timeGrafana in space: Monitoring Japan's SLIM moon lander  in real time
Grafana in space: Monitoring Japan's SLIM moon lander in real timeSatoshi NAKAHIRA
 
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptxUnlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptxanandsmhk
 
Animal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptxAnimal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptxUmerFayaz5
 
SOLUBLE PATTERN RECOGNITION RECEPTORS.pptx
SOLUBLE PATTERN RECOGNITION RECEPTORS.pptxSOLUBLE PATTERN RECOGNITION RECEPTORS.pptx
SOLUBLE PATTERN RECOGNITION RECEPTORS.pptxkessiyaTpeter
 
Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​  ​Nanoparticles synthesis and characterization​  ​
Nanoparticles synthesis and characterization​ ​kaibalyasahoo82800
 
Is RISC-V ready for HPC workload? Maybe?
Is RISC-V ready for HPC workload? Maybe?Is RISC-V ready for HPC workload? Maybe?
Is RISC-V ready for HPC workload? Maybe?Patrick Diehl
 
Behavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdfBehavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdfSELF-EXPLANATORY
 
A relative description on Sonoporation.pdf
A relative description on Sonoporation.pdfA relative description on Sonoporation.pdf
A relative description on Sonoporation.pdfnehabiju2046
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxAArockiyaNisha
 
Genomic DNA And Complementary DNA Libraries construction.
Genomic DNA And Complementary DNA Libraries construction.Genomic DNA And Complementary DNA Libraries construction.
Genomic DNA And Complementary DNA Libraries construction.k64182334
 
Ahmedabad Call Girls Service 9537192988 can satisfy every one of your dreams
Ahmedabad Call Girls Service 9537192988 can satisfy every one of your dreamsAhmedabad Call Girls Service 9537192988 can satisfy every one of your dreams
Ahmedabad Call Girls Service 9537192988 can satisfy every one of your dreamsoolala9823
 
GFP in rDNA Technology (Biotechnology).pptx
GFP in rDNA Technology (Biotechnology).pptxGFP in rDNA Technology (Biotechnology).pptx
GFP in rDNA Technology (Biotechnology).pptxAleenaTreesaSaji
 
Biopesticide (2).pptx .This slides helps to know the different types of biop...
Biopesticide (2).pptx  .This slides helps to know the different types of biop...Biopesticide (2).pptx  .This slides helps to know the different types of biop...
Biopesticide (2).pptx .This slides helps to know the different types of biop...RohitNehra6
 
Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |aasikanpl
 
Orientation, design and principles of polyhouse
Orientation, design and principles of polyhouseOrientation, design and principles of polyhouse
Orientation, design and principles of polyhousejana861314
 
Module 4: Mendelian Genetics and Punnett Square
Module 4:  Mendelian Genetics and Punnett SquareModule 4:  Mendelian Genetics and Punnett Square
Module 4: Mendelian Genetics and Punnett SquareIsiahStephanRadaza
 
TOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physicsTOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physicsssuserddc89b
 

Recently uploaded (20)

Natural Polymer Based Nanomaterials
Natural Polymer Based NanomaterialsNatural Polymer Based Nanomaterials
Natural Polymer Based Nanomaterials
 
Artificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C PArtificial Intelligence In Microbiology by Dr. Prince C P
Artificial Intelligence In Microbiology by Dr. Prince C P
 
Boyles law module in the grade 10 science
Boyles law module in the grade 10 scienceBoyles law module in the grade 10 science
Boyles law module in the grade 10 science
 
Grafana in space: Monitoring Japan's SLIM moon lander in real time
Grafana in space: Monitoring Japan's SLIM moon lander  in real timeGrafana in space: Monitoring Japan's SLIM moon lander  in real time
Grafana in space: Monitoring Japan's SLIM moon lander in real time
 
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptxUnlocking  the Potential: Deep dive into ocean of Ceramic Magnets.pptx
Unlocking the Potential: Deep dive into ocean of Ceramic Magnets.pptx
 
Animal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptxAnimal Communication- Auditory and Visual.pptx
Animal Communication- Auditory and Visual.pptx
 
SOLUBLE PATTERN RECOGNITION RECEPTORS.pptx
SOLUBLE PATTERN RECOGNITION RECEPTORS.pptxSOLUBLE PATTERN RECOGNITION RECEPTORS.pptx
SOLUBLE PATTERN RECOGNITION RECEPTORS.pptx
 
Nanoparticles synthesis and characterization​ ​
Nanoparticles synthesis and characterization​  ​Nanoparticles synthesis and characterization​  ​
Nanoparticles synthesis and characterization​ ​
 
Is RISC-V ready for HPC workload? Maybe?
Is RISC-V ready for HPC workload? Maybe?Is RISC-V ready for HPC workload? Maybe?
Is RISC-V ready for HPC workload? Maybe?
 
Behavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdfBehavioral Disorder: Schizophrenia & it's Case Study.pdf
Behavioral Disorder: Schizophrenia & it's Case Study.pdf
 
A relative description on Sonoporation.pdf
A relative description on Sonoporation.pdfA relative description on Sonoporation.pdf
A relative description on Sonoporation.pdf
 
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptxPhysiochemical properties of nanomaterials and its nanotoxicity.pptx
Physiochemical properties of nanomaterials and its nanotoxicity.pptx
 
Genomic DNA And Complementary DNA Libraries construction.
Genomic DNA And Complementary DNA Libraries construction.Genomic DNA And Complementary DNA Libraries construction.
Genomic DNA And Complementary DNA Libraries construction.
 
Ahmedabad Call Girls Service 9537192988 can satisfy every one of your dreams
Ahmedabad Call Girls Service 9537192988 can satisfy every one of your dreamsAhmedabad Call Girls Service 9537192988 can satisfy every one of your dreams
Ahmedabad Call Girls Service 9537192988 can satisfy every one of your dreams
 
GFP in rDNA Technology (Biotechnology).pptx
GFP in rDNA Technology (Biotechnology).pptxGFP in rDNA Technology (Biotechnology).pptx
GFP in rDNA Technology (Biotechnology).pptx
 
Biopesticide (2).pptx .This slides helps to know the different types of biop...
Biopesticide (2).pptx  .This slides helps to know the different types of biop...Biopesticide (2).pptx  .This slides helps to know the different types of biop...
Biopesticide (2).pptx .This slides helps to know the different types of biop...
 
Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |
Call Us ≽ 9953322196 ≼ Call Girls In Mukherjee Nagar(Delhi) |
 
Orientation, design and principles of polyhouse
Orientation, design and principles of polyhouseOrientation, design and principles of polyhouse
Orientation, design and principles of polyhouse
 
Module 4: Mendelian Genetics and Punnett Square
Module 4:  Mendelian Genetics and Punnett SquareModule 4:  Mendelian Genetics and Punnett Square
Module 4: Mendelian Genetics and Punnett Square
 
TOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physicsTOPIC 8 Temperature and Heat.pdf physics
TOPIC 8 Temperature and Heat.pdf physics
 

Robustness and evolution of novel functions in programmable hardware

  • 1. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary . . . .. . . Robustness and evolution of novel functions in programmable hardware Karthik Raman Andreas Wagner Lab Institute of Evolutionary Biology and Environmental Studies University of Zürich April 1, 2011 1 / 40
  • 2. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Outline . . .1 Introduction Robustness and Evolvability Neutral Networks . . .2 Genotype Space and Neutral Networks Genotype space Phenotype space Neutral networks . . .3 Robustness and Evolvability Robustness of circuits Evolving new functions . . .4 Summary 2 / 40
  • 3. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Biological systems vs. Man-made systems . Biological systems .. . . .. . . Shaped through the forces of mutation and natural selection Survive onslaught of disruptive agents — hostile environments/random mutations: robust Show a remarkable ability to adapt and evolve novel properties through such random mutations: evolvable . Man-made systems .. . . .. . . Product of rational design, rather than biological evolution Generally not as robust: often fragile — modification/removal of components results in catastrophic failure Their ability to acquire novel and useful features through random change is limited 3 / 40
  • 4. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Biological systems vs. Man-made systems . Biological systems .. . . .. . . Shaped through the forces of mutation and natural selection Survive onslaught of disruptive agents — hostile environments/random mutations: robust Show a remarkable ability to adapt and evolve novel properties through such random mutations: evolvable . Man-made systems .. . . .. . . Product of rational design, rather than biological evolution Generally not as robust: often fragile — modification/removal of components results in catastrophic failure Their ability to acquire novel and useful features through random change is limited 3 / 40
  • 5. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Biological systems vs. Man-made systems . Biological systems .. . . .. . . Shaped through the forces of mutation and natural selection Survive onslaught of disruptive agents — hostile environments/random mutations: robust Show a remarkable ability to adapt and evolve novel properties through such random mutations: evolvable . Man-made systems .. . . .. . . Product of rational design, rather than biological evolution Generally not as robust: often fragile — modification/removal of components results in catastrophic failure Their ability to acquire novel and useful features through random change is limited 3 / 40
  • 6. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Robustness Fundamental feature of most biological systems — ability to continue normal function in the face of perturbations Man-made systems — generally not very robust to failure Design of fault-tolerant systems is receiving increased attention 4 / 40
  • 7. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Evolvability “The ability to produce phenotypic diversity, novel solutions to the problems faced by organisms and evolutionary innovations” Common feature of biological systems — novel functions, which help an organism survive and reproduce, are acquired through genetic change. The need for adaptation in artificial systems is being recognised — ability to learn and adapt to new situations, e. g. autonomous robots 5 / 40
  • 8. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Evolvable Hardware Principles of evolution have been applied in computer science for solving many complex optimisation problems — evolutionary computationa implements aspects such as random variation, reproduction, selection, . . . Artificial evolution can automatically generate designs of digital circuits, as well as circuits that are robust to noise and faultsb Evolvable hardware = evolutionary algorithms applied to electronic circuits and devices a Foster JA (2001) Nat Rev Genet 2:428–436 b Hartmann M & Haddow P (2004) IEE Proceedings - Computers and Digital Techniques 151:287–294 6 / 40
  • 9. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Key Questions How ‘robust’ is a typical circuit to changes in the wiring/configuration? Do neutral networks exist in this circuit space? Can circuits with significantly different configuration compute the same Boolean function? Does the organisation of neutral networks facilitate the adoption of novel phenotypes (logic function computations) through small numbers of gate changes? 7 / 40
  • 10. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets 8 / 40
  • 11. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets Neutral networks — Genotypes sharing the same phenotype 8 / 40
  • 12. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets Neutral networks — Genotypes sharing the same phenotype 8 / 40
  • 13. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets Neutral networks — Genotypes sharing the same phenotype 8 / 40
  • 14. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets Neutral networks — Genotypes sharing the same phenotype 8 / 40
  • 15. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets Neutral networks — Genotypes sharing the same phenotype 8 / 40
  • 16. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks Genotype space Nodes: Genotypes Edges: mutation/simple genetic change Phenotypes mapped to genotypes Largest neutral network Smaller neutral network Smaller and more fragmented neutral set Disconnected multiple neutral sets Neutral networks — Genotypes sharing the same phenotype The genotype space is covered with multiple neutral sets/networks Neutral networks have implications for robustness/evolvability 8 / 40
  • 17. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Genotype vs. Phenotype Genotype evolves through mutation and is inherited; selection acts on phenotype Genotype–phenotype mapping — required to distinguish neutral networks Mapping may not be straightforward always System Genotype Phenotype RNA/Proteins Sequence Structure Regulatory networks Regulatory interactions Gene expression patterns Circadian oscillators Regulatory interactions Oscillations Digital circuits Circuit components/wiring Boolean function computed Protein–protein interactions Network topology ? Signal transduction Network topology ? 9 / 40
  • 18. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral networks vs. Robustness/Evolvability Neutral networks can be traversed in small evolutionary steps, through simple changes/mutations Larger neutral network ⇒ higher robustness, scope for evolutionary innovation … 10 / 40
  • 19. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks vs. Robustness Time High robustness Low robustness Low robustness: more deleterious mutations High robustness: likely to encounter more novel phenotypes 11 / 40
  • 20. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks vs. Robustness Time High robustness Low robustness Low robustness: more deleterious mutations High robustness: likely to encounter more novel phenotypes 11 / 40
  • 21. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral Networks vs. Robustness Time High robustness Low robustness —Wagner A (2008a) Nat Rev Genet 9:965–974 Low robustness: more deleterious mutations High robustness: likely to encounter more novel phenotypes 11 / 40
  • 22. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Robustness vs. Evolvability Robustness and evolvability — both correlate with neutral network size/connectivity Robust phenotypes tend to have higher evolvability Low Robustness Low Innovation High Robustness Low Innovation High Robustness High Innovation —Ciliberti S et al. (2007) Proc Natl Acad Sci U S A 104:13591–13596 Populations evolving on larger neutral networks have greater access to variation* 12 / 40
  • 23. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Outline . . .1 Introduction Robustness and Evolvability Neutral Networks . . .2 Genotype Space and Neutral Networks Genotype space Phenotype space Neutral networks . . .3 Robustness and Evolvability Robustness of circuits Evolving new functions . . .4 Summary 13 / 40
  • 24. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Field Programmable Gate Arrays (FPGAs) I N P U T S 1 2 3 4 6 7 O U T P U T S 5 8 9 10 13 12 11 OR A B Y AND A B Y NAND A B Y NOR A B Y XOR A B Y Y=A+B Y=A×B Y=AÅB ___ Y=(A×B) ____ Y=(A+B) n × m array of logic gates Configuration can be dynamically reprogrammed: ‘field programmable’ Maps a Boolean input of nI variables to an output of no variables: f : {0, 1}nI → {0, 1}no 14 / 40
  • 25. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Generic FPGA inputs outputs nI inputs, mapped to no outputs All nI inputs must be used in column 1 n × m array of logic gates, Lij, each with two inputs Inter-connections are feed-forward: a gate in column i can take an input from any of the previous levels no outputs can come from any of the mn gate outputs 15 / 40
  • 26. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Representation of FPGAs I N P U T S 1 2 3 4 6 7 O U T P U T S 5 8 9 10 13 12 11 4 2 3 L11 1 1 5 L12 3 2 2 L13 2 2 5 L21 1 7 2 L22 5 2 3 L23 10 5 3 L31 3 3 1 L32 6 6 4 L33 11 12 10 13 Outputs Size of representation = 3mn + no = diameter of circuit space [Similar to Cartesian Genetic Programming — Miller & Thomson (2000)] 16 / 40
  • 27. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Genotype space: Circuits with various configurations I N P U T S 1 2 4 53 6 O U T P U T S I N P U T S 1 2 4 53 6 O U T P U T S I N P U T S 1 2 4 53 6 O U T P U T S I N P U T S 1 2 4 53 6 O U T P U T S I N P U T S 1 2 4 53 6 O U T P U T S C1 C2 C3C4C5 C6 I N P U T S 1 2 4 53 6 O U T P U T S Every node in genotype space is a circuit (of a particular size) Neighbours vary by a small elementary change in: (a) Internal wiring (C2), (b) Logic function of one of the gates (C3), (c) Input mapping (C5), or (d) Output mapping (C6) d(C4, C1) = 1 ⇒ not neighbours 17 / 40
  • 28. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Phenotype Space: Boolean functions Every circuit maps a Boolean input of nI bits to an output of no bits: f : {0, 1}nI → {0, 1}no For example, i j k l ⇒ j k l i Circular [Left] Shift i j k l ⇒ 0 i j k [Unsigned] Right Shift I1 I2 I3 I4 O1 O2 O3 O4 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 0 0 0 1 1 0 1 1 0 0 1 0 0 1 0 0 0 0 1 0 1 1 0 1 0 0 1 1 0 1 1 0 0 0 1 1 1 1 1 1 0 1 0 0 0 0 0 0 1 1 0 0 1 0 0 1 1 1 0 1 0 0 1 0 1 1 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 0 1 1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 Circular left shift truth table Total Boolean functions = 2entries in truth table = 264 = 1.84 × 1019 18 / 40
  • 29. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Sampling circuit space Sample size = 2 × 107 Circuit space sampled uniformly FPGA size Genotype space size Functions observed in sample 3 × 3 1.08 ×1024 1.59 ×107 4 × 4 8.43 ×1045 1.74 ×107 5 × 5 2.90 ×1076 1.80 ×107 6 × 6 6.05 ×10116 1.83 ×107 Total Boolean functions = 264 = 1.84 × 1019 Small fraction of total Boolean functions observed: ≈ 10−12 Most observed functions are computed only by one circuit in the sample Few functions are computed by large number of circuits 19 / 40
  • 30. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Majority of functions are rare 10 0 10 5 10 −8 10 −7 10 −6 10 −5 10 −4 Rank of logic function (decreasing size of circuit sets) Frequencyoflogicfunction Some functions appear over a 1000 times Long tail: Huge number of functions appear only once Functions such as right shift/circular shift do not appear in the sample of 2 × 107 20 / 40
  • 31. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neutral networks in circuit space Circuits computing the same function can be reached through function-preserving random walks in circuit space ⇒ Circuit space is spanned by large neutral networks 0 0.2 0.4 0.6 0.8 1 10 0 10 1 10 2 10 3 Maximal circuit distance (D) Numberoffunctions 4x4 Circuits 10 −7 10 −6 10 −5 10 −4 0.3 0.5 0.7 1 Frequency of the logic functionMaximalcircuitdistance(D) 4x4 Circuits Spearman’s r = 0.56; p < 10−300 ; n = 1000 A function-preserving random walk in circuit space can take us very far away from the initial circuit ⇒ Neutral networks have very large diameter 21 / 40
  • 32. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Diverse circuits can compute the same function I N P U T S 1 2 3 4 6 7 O U T P U T S 5 8 9 10 13 12 11 4 2 3 1 1 5 3 2 2 2 2 5 1 7 2 5 2 3 10 5 3 3 3 1 6 6 4 11 12 10 13 I N P U T S 1 2 3 4 6 7 O U T P U T S 5 8 9 10 13 12 11 2 3 1 4 4 1 1 1 3 5 3 4 7 6 1 7 3 1 1 1 2 1 5 2 5 2 2 13 10 9 11 Both circuits compute circular shift Differ in every logic gate, wiring, input & output mapping Thus, the diameters of neutral networks can be quite large 22 / 40
  • 33. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Outline . . .1 Introduction Robustness and Evolvability Neutral Networks . . .2 Genotype Space and Neutral Networks Genotype space Phenotype space Neutral networks . . .3 Robustness and Evolvability Robustness of circuits Evolving new functions . . .4 Summary 23 / 40
  • 34. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Robustness to change in configuration 10 −7 10 −6 10 −5 10 −4 0 0.2 0.4 0.6 0.8 1 Frequency of the logic function Robustnesstoconfigurationchange 4x4 Circuits Spearman’s r = 0.32; p < 10 −300 0 0.2 0.4 0.6 0.8 1 0 50 100 Circular shift (3x3) 0 0.2 0.4 0.6 0.8 1 0 20 40 Numberofcircuits Circular shift (4x4) 0 0.2 0.4 0.6 0.8 1 0 20 40 Fraction of neutral neighbors Circular shift (6x6) Robustness = fraction of neutral neighbours Robustness distribution is quite broad Larger circuits are more robust (regardless of function) 24 / 40
  • 35. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Robustness to gate failure 10 −7 10 −6 10 −5 10 −4 0 0.2 0.4 0.6 0.8 1 Frequency of the logic function Robustnesstogatefailure 4x4 Circuits 0 0.2 0.4 0.6 0.8 1 0 50 Circular shift (3x3) 0 0.2 0.4 0.6 0.8 1 0 20 40 Numberofcircuits Circular shift (4x4) 0 0.2 0.4 0.6 0.8 1 0 20 40 Failure robustness Circular shift (6x6) Gate failures are common in digital circuits Types of failures considered: zero output (and inverted output) Failure robustness = fraction of single gate failures that do not affect the computed function Failure robustness distribution is also quite broad Larger circuits are more robust to gate failure 25 / 40
  • 36. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Evolving new functions On large neutral networks, genotypes can change substantially without changing phenotype Phenotypes in different neighbourhoods of a neutral network can be quite different In biological systems, this feature facilitates the exploration of new phenotypesab For circuits, this will have implications for the ease with which evolvable hardware can acquire new functions a Ciliberti S et al. (2007) Proc Natl Acad Sci U S A 104:13591–13596 b Wagner A (2008b) Proc Biol Sci 275:91–100 26 / 40
  • 37. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neighbourhood of evolving circuits 0 500 1000 1500 2000 0 500 1000 1500 2000 2500 3000 Random walk steps Cumulativenovelfunctions encounteredinneighborhood 0 1000 2000 0 1000 2000 3000 f = 5.00 ⋅ 10 −8 Cumulativenovelfunctions encounteredinneighborhood 0 1000 2000 0 1000 2000 3000 f = 5.00 ⋅ 10 −8 0 1000 2000 0 1000 2000 3000 f = 3.00 ⋅ 10 −7 0 1000 2000 0 1000 2000 3000 f = 4.65 ⋅ 10 −6 0 1000 2000 0 1000 2000 3000 f = 6.35 ⋅ 10−6 Random walk steps Cumulativenovelfunctions encounteredinneighborhood 0 1000 2000 0 1000 2000 3000 f = 9.40 ⋅ 10−6 Random walk steps 0 1000 2000 0 1000 2000 3000 f = 1.24 ⋅ 10−5 Random walk steps 0 1000 2000 0 1000 2000 3000 f = 5.09 ⋅ 10−5 Random walk steps As we explore the neutral network (through a function-preserving random walk), a large number of new phenotypes are encountered The number of new phenotypes keeps increasing, as we prolong the random walk This property holds for a large number of functions Indicates the high accessibility of innovation in the neighbourhood 27 / 40
  • 38. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neighbourhood of evolving circuits What is the fraction of new phenotypes encountered in the neighbourhood of any circuit? With reference to a starting circuit C0, u(C0, Ci) = 1 − |N0 ∩ Ni| |N0 ∪ Ni| 28 / 40
  • 39. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neighbourhood of evolving circuits What is the fraction of new phenotypes encountered in the neighbourhood of any circuit? With reference to a starting circuit C0, u(C0, Ci) = 1 − |N0 ∩ Ni| |N0 ∪ Ni| G1 G2 shared unique u(G1, G2) = = 5 8 = 0.625 28 / 40
  • 40. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Neighbourhood of evolving circuits 0 50 100 150 200 0 0.2 0.4 0.6 0.8 1 Random walk steps Fractionofnewfunctions inneighborhood 0 1000 2000 0 0.5 1 f = 5.00 ⋅ 10 −8 Fractionofnewfunctions inneighborhood 0 1000 2000 0 0.5 1 f = 5.00 ⋅ 10 −8 0 1000 2000 0 0.5 1 f = 3.00 ⋅ 10 −7 0 1000 2000 0 0.5 1 f = 4.65 ⋅ 10 −6 0 1000 2000 0 0.5 1 f = 6.35 ⋅ 10−6 Random walk steps Fractionofnewfunctions inneighborhood 0 1000 2000 0 0.5 1 f = 9.40 ⋅ 10−6 Random walk steps 0 1000 2000 0 0.5 1 f = 1.24 ⋅ 10−5 Random walk steps 0 1000 2000 0 0.5 1 f = 5.09 ⋅ 10−5 Random walk steps Beyond the distance of ≈ one network diameter, over 80% of the functions found in the neighbourhood are new At increasing distances from a circuit, a very large number of new phenotypes are accessible; this property holds for a large number of functions 29 / 40
  • 41. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Fraction of new phenotypes at the end of a random walk 10 −7 10 −6 10 −5 10 −4 0 0.2 0.4 0.6 0.8 1 Frequency of the logic function Fractionofnewfunctionsin neighborhoodatendofrandomwalk 4x4 Circuits 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 Fraction of new functions in neighborhood at the end of random walk Numberofcircuits Circular shift (4x4) A lot of innovation is accessible in the neighbourhood, once we walk sufficiently far awaya from a starting circuit . . . a 2,000 steps, in this case 30 / 40
  • 42. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Proximity of neutral networks How far must one travel in circuit space from one neutral set, to find another neutral set? 31 / 40
  • 43. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Proximity of neutral networks How far must one travel in circuit space from one neutral set, to find another neutral set? 0 0.2 0.4 0.6 0.8 1 0 50 100 150 200 Minimum distance between circuits computing right shift and circular shift (as fraction of circuit space diameter) Pairsofcircuits 0 0.2 0.4 0.6 0.8 1 0 200 400 600 800 Minimum distance between neutral networks (as fraction of circuit space diameter) Pairsofneutralnetworksd(CS, RS) ≤ 3! It is possible to discover new functions through a relatively small number of elementary changes 31 / 40
  • 44. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Similarity with Biological Systems The system of circuits studied show intriguing similarities with biological systems: Existence of large highly connected neutral networks Large number of diverse genotypes adopt the same phenotype Robustness to change in configurations Large number of novel phenotypes accessible in the neighbourhood of evolving genotypes Proximity of different neutral networks Importance of “junk” non-functional components … 32 / 40
  • 45. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Role of ‘non-functional components’ Many circuits have non-functional gates, system parts that are not involved in the computation a given circuit carries out In biological systems, many amino acids in a proteinab , many regulatory interactions in a gene regulation circuitc , and many metabolic reactions in a metabolic reaction networkd may appear as ‘non-functional’ or ‘dispensable’ Tempting to call such system parts ‘junk’ parts — however, such parts play a crucial role for evolvability, and it is precisely their ability to vary freely in some environments that allows biological systems to evolve novel phenotypes For example, in laboratory evolution experiments, proteins with new function evolve often through changes that do not affect the protein’s principal functionb Circuits of a minimal size may have the merit of computing a function in an elegant and simple way — at the same time, they would be utterly unevolvable Evolvability comes at the price of high complexity a Bloom JD et al. (2005) Proc Natl Acad Sci U S A 102:606–611 b Aharoni A et al. (2005) Nat Genet 37:73–76 c Ciliberti S et al. (2007) Proc Natl Acad Sci U S A 104:13591–13596 d Matias Rodrigues JF & Wagner A (2009) PLoS Comput Biol 5:e1000613 33 / 40
  • 46. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Outline . . .1 Introduction Robustness and Evolvability Neutral Networks . . .2 Genotype Space and Neutral Networks Genotype space Phenotype space Neutral networks . . .3 Robustness and Evolvability Robustness of circuits Evolving new functions . . .4 Summary 34 / 40
  • 47. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Limitations Many other factors decide the suitability of a particular circuit, e. g. power consumption, robustness to temperature variations, and trade-offs between functional flexibility and performance Fine differences exist between two silicon chips – circuits evolved on one silicon chip are not guaranteed to work on anothera ! Differences exist, between software simulation and hardware evolution Some properties of our (or any other) study system may depend on the choice of representation for a circuit’s architecture Possible major computational issues with very large circuit sizes Due to the astronomical numbers of circuits and functions, one needs to resort to sampling to understand circuit space — not limiting if one is interested in generic properties of this space a Thompson A (1995) In: F Morán, A Moreno, JJ Merelo, & P Chacon (eds.), Advances in Artificial Life: Proc. 3rd Eur. Conf. on Artificial Life (ECAL95), volume 929 of LNAI, pages 640–656. Springer-Verlag 35 / 40
  • 48. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Summary A random sampling of circuit space reveals that majority of the functions are computed by very few circuits Circuits computing the same function form large connected neutral networks with large diameters Diverse circuits can compute the same function Larger circuits tend to be more robust; the distribution of robustness is broad for a given function and size, the most robust configuration may be useful in applications A number of new functions are accessible in the neighbourhood of evolving circuits evolvable circuits may find application in the design of adaptive devices Neutral networks are also located close together in space access to new functions through few changes ‘minimal reconfiguration’ may be particularly useful/efficient 36 / 40
  • 49. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Summary A random sampling of circuit space reveals that majority of the functions are computed by very few circuits Circuits computing the same function form large connected neutral networks with large diameters Diverse circuits can compute the same function Larger circuits tend to be more robust; the distribution of robustness is broad for a given function and size, the most robust configuration may be useful in applications A number of new functions are accessible in the neighbourhood of evolving circuits evolvable circuits may find application in the design of adaptive devices Neutral networks are also located close together in space access to new functions through few changes ‘minimal reconfiguration’ may be particularly useful/efficient 36 / 40
  • 50. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Summary A random sampling of circuit space reveals that majority of the functions are computed by very few circuits Circuits computing the same function form large connected neutral networks with large diameters Diverse circuits can compute the same function Larger circuits tend to be more robust; the distribution of robustness is broad for a given function and size, the most robust configuration may be useful in applications A number of new functions are accessible in the neighbourhood of evolving circuits evolvable circuits may find application in the design of adaptive devices Neutral networks are also located close together in space access to new functions through few changes ‘minimal reconfiguration’ may be particularly useful/efficient 36 / 40
  • 51. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Summary A random sampling of circuit space reveals that majority of the functions are computed by very few circuits Circuits computing the same function form large connected neutral networks with large diameters Diverse circuits can compute the same function Larger circuits tend to be more robust; the distribution of robustness is broad for a given function and size, the most robust configuration may be useful in applications A number of new functions are accessible in the neighbourhood of evolving circuits evolvable circuits may find application in the design of adaptive devices Neutral networks are also located close together in space access to new functions through few changes ‘minimal reconfiguration’ may be particularly useful/efficient 36 / 40
  • 52. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Summary A random sampling of circuit space reveals that majority of the functions are computed by very few circuits Circuits computing the same function form large connected neutral networks with large diameters Diverse circuits can compute the same function Larger circuits tend to be more robust; the distribution of robustness is broad for a given function and size, the most robust configuration may be useful in applications A number of new functions are accessible in the neighbourhood of evolving circuits evolvable circuits may find application in the design of adaptive devices Neutral networks are also located close together in space access to new functions through few changes ‘minimal reconfiguration’ may be particularly useful/efficient 36 / 40
  • 53. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Outlook: Exploiting robustness and evolvability . In technological systems .. . . .. . . For any desired Boolean function, it may be beneficial to use the most robust circuit (one that is resistant to a number of gate failures as well as configuration changes) of a particular size Circuits that have access to a greater amount of innovation (diverse Boolean functions), may be useful in designing systems with adaptive behaviour . In biology … .. . . .. . . Such circuits can also be viewed as generalised models of signalling networks/gene regulatory networks Such circuits also find applications in synthetic biology, for the design of synthetic circuitsa a Marchisio MA & Stelling J (2011) PLoS Comput Biol 7:e1001083+ 37 / 40
  • 54. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Not all circuits computing the same function are equal … I N P U T S 1 2 3 4 6 7 O U T P U T S 5 8 Circuit A Robustness = 0.0556 I N P U T S 1 2 3 4 O U T P U T S 1915117 2016128 1814106 171395 Circuit B Robustness = 0.5725 …is complexity the price for evolvability? 38 / 40
  • 55. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary Thank you 39 / 40
  • 56. . . . . . . . . . . Introduction . . . . . . . . . Genotype Space and Neutral Networks . . . . . . . . . . Robustness and Evolvability . . . . . . Summary 40 / 40