The document discusses the concept of schemas in genetic algorithms (GAs), defining schemas as templates that identify subsets of similar strings within a fixed-length binary alphabet. It elaborates on the properties of schemas, including their order and defining length, and explores the probabilities of schemas surviving various genetic operations such as mutation and crossover. Additionally, the document critiques the schema theorem and presents cases to illustrate the impact of mutations on the survival of these schemas.

1. Artificial Intelligence
Holland’s GA Schema Theorem
Andres Mendez-Vazquez
April 7, 2015
1 / 37

2. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
2 / 37

3. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
3 / 37

4. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

5. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

6. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

7. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

8. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

9. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

10. Introduction
Consider the Canonical GA
Binary alphabet.
Fixed length individuals of equal length, l.
Fitness Proportional Selection.
Single Point Crossover (1X).
Gene wise mutation i.e. mutate gene by gene.
Definition 1 - Schema H
A schema H is a template that identifies a subset of strings with
similarities at certain string positions.
Schemata are a special case of a natural open set of a product
topology.
4 / 37

11. Introduction
Definition 2
If A denotes the alphabet of genes, then A ∪ ∗ is the schema alphabet,
where * is the ‘wild card’ symbol matching any gene value.
Example: For A ∈ {0, 1, ∗} where ∗ ∈ {0, 1}.
5 / 37

12. Example
The Schema H = [0 1 ∗ 1 *] generates the following individuals
0 1 * 1 *
0 1 0 1 0
0 1 0 1 1
0 1 1 1 0
0 1 1 1 1
Not all schemas say the same
Schema [ 1 ∗ ∗ ∗ ∗ ∗ ∗] has less information than [ 0 1 ∗ ∗ 1 1 0].
It is more!!!
[ 1 ∗ ∗ ∗ ∗ ∗ 0] span the entire length of an individual, but
[ 1 ∗ 1 ∗ ∗ ∗ ∗] does not.
6 / 37

13. Example
The Schema H = [0 1 ∗ 1 *] generates the following individuals
0 1 * 1 *
0 1 0 1 0
0 1 0 1 1
0 1 1 1 0
0 1 1 1 1
Not all schemas say the same
Schema [ 1 ∗ ∗ ∗ ∗ ∗ ∗] has less information than [ 0 1 ∗ ∗ 1 1 0].
It is more!!!
[ 1 ∗ ∗ ∗ ∗ ∗ 0] span the entire length of an individual, but
[ 1 ∗ 1 ∗ ∗ ∗ ∗] does not.
6 / 37

14. Example
The Schema H = [0 1 ∗ 1 *] generates the following individuals
0 1 * 1 *
0 1 0 1 0
0 1 0 1 1
0 1 1 1 0
0 1 1 1 1
Not all schemas say the same
Schema [ 1 ∗ ∗ ∗ ∗ ∗ ∗] has less information than [ 0 1 ∗ ∗ 1 1 0].
It is more!!!
[ 1 ∗ ∗ ∗ ∗ ∗ 0] span the entire length of an individual, but
[ 1 ∗ 1 ∗ ∗ ∗ ∗] does not.
6 / 37

15. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
7 / 37

16. Schema Order and Length
Definition 3 - Schema Order o (H)
Schema order, o, is the number of non “*” genes in schema H.
Example: o(***11*1)=3.
Definition 3 – Schema Defining Length, δ (H).
Schema Defining Length, δ(H), is the distance between first and last non
“*” gene in schema H.
Example: δ(***11*1)=7-4=3.
Notes
Given an alphabet A with|A| = k, then there are (k + 1)l
possible
schemas of length l.
8 / 37

17. Schema Order and Length
Definition 3 - Schema Order o (H)
Schema order, o, is the number of non “*” genes in schema H.
Example: o(***11*1)=3.
Definition 3 – Schema Defining Length, δ (H).
Schema Defining Length, δ(H), is the distance between first and last non
“*” gene in schema H.
Example: δ(***11*1)=7-4=3.
Notes
Given an alphabet A with|A| = k, then there are (k + 1)l
possible
schemas of length l.
8 / 37

18. Schema Order and Length
Definition 3 - Schema Order o (H)
Schema order, o, is the number of non “*” genes in schema H.
Example: o(***11*1)=3.
Definition 3 – Schema Defining Length, δ (H).
Schema Defining Length, δ(H), is the distance between first and last non
“*” gene in schema H.
Example: δ(***11*1)=7-4=3.
Notes
Given an alphabet A with|A| = k, then there are (k + 1)l
possible
schemas of length l.
8 / 37

19. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
9 / 37

20. Probabilities of belonging to a Schema H
What do we want?
The probability that individual h is from schema H:
P (h ∈ H)
We need the following probabilities
Pdistruption(H, 1X) = probability of schema being disrupted due to
crossover.
Pdisruption (H, mutation) =probability of schema being disrupted due
to mutation
Pcrossover (H survive)
10 / 37

21. Probabilities of belonging to a Schema H
What do we want?
The probability that individual h is from schema H:
P (h ∈ H)
We need the following probabilities
Pdistruption(H, 1X) = probability of schema being disrupted due to
crossover.
Pdisruption (H, mutation) =probability of schema being disrupted due
to mutation
Pcrossover (H survive)
10 / 37

22. Probabilities of belonging to a Schema H
What do we want?
The probability that individual h is from schema H:
P (h ∈ H)
We need the following probabilities
Pdistruption(H, 1X) = probability of schema being disrupted due to
crossover.
Pdisruption (H, mutation) =probability of schema being disrupted due
to mutation
Pcrossover (H survive)
10 / 37

23. Probability of Disruption
Consider now
The CGA using
fitness proportionate parent selection
on-point crossover (1X)
bitwise mutation with probability Pm
Genotypes of length l
The Schema could be disrupted if the cross over falls between the
ends
Pdistruption(H, 1X) =
δ(H)
(l − 1)
(1)
0 1 0 0 1 0
11 / 37

24. Probability of Disruption
Consider now
The CGA using
fitness proportionate parent selection
on-point crossover (1X)
bitwise mutation with probability Pm
Genotypes of length l
The Schema could be disrupted if the cross over falls between the
ends
Pdistruption(H, 1X) =
δ(H)
(l − 1)
(1)
0 1 0 0 1 0
11 / 37

25. Probability of Disruption
Consider now
The CGA using
fitness proportionate parent selection
on-point crossover (1X)
bitwise mutation with probability Pm
Genotypes of length l
The Schema could be disrupted if the cross over falls between the
ends
Pdistruption(H, 1X) =
δ(H)
(l − 1)
(1)
0 1 0 0 1 0
11 / 37

26. Probability of Disruption
Consider now
The CGA using
fitness proportionate parent selection
on-point crossover (1X)
bitwise mutation with probability Pm
Genotypes of length l
The Schema could be disrupted if the cross over falls between the
ends
Pdistruption(H, 1X) =
δ(H)
(l − 1)
(1)
0 1 0 0 1 0
11 / 37

27. Probability of Disruption
Consider now
The CGA using
fitness proportionate parent selection
on-point crossover (1X)
bitwise mutation with probability Pm
Genotypes of length l
The Schema could be disrupted if the cross over falls between the
ends
Pdistruption(H, 1X) =
δ(H)
(l − 1)
(1)
0 1 0 0 1 0
11 / 37

28. Probability of Disruption
Consider now
The CGA using
fitness proportionate parent selection
on-point crossover (1X)
bitwise mutation with probability Pm
Genotypes of length l
The Schema could be disrupted if the cross over falls between the
ends
Pdistruption(H, 1X) =
δ(H)
(l − 1)
(1)
0 1 0 0 1 0
11 / 37

29. Why?
Given that you have
δ(H) = the distance between first and last non “*”
last position in Genotype - first position in Genotype = l − 1
Case I
δ(H) = 1, when the positions of the non “*” are next to each other
Case II
δ(H) = l − 1, when the positions of the non “*” are in the extremes
12 / 37

30. Why?
Given that you have
δ(H) = the distance between first and last non “*”
last position in Genotype - first position in Genotype = l − 1
Case I
δ(H) = 1, when the positions of the non “*” are next to each other
Case II
δ(H) = l − 1, when the positions of the non “*” are in the extremes
12 / 37

31. Why?
Given that you have
δ(H) = the distance between first and last non “*”
last position in Genotype - first position in Genotype = l − 1
Case I
δ(H) = 1, when the positions of the non “*” are next to each other
Case II
δ(H) = l − 1, when the positions of the non “*” are in the extremes
12 / 37

32. Why?
Given that you have
δ(H) = the distance between first and last non “*”
last position in Genotype - first position in Genotype = l − 1
Case I
δ(H) = 1, when the positions of the non “*” are next to each other
Case II
δ(H) = l − 1, when the positions of the non “*” are in the extremes
12 / 37

33. Remarks about Mutation
Observation about Mutation
Mutation is applied gene by gene.
In order for schema H to survive, all non * genes in the schema much
remain unchanged.
Thus
Probability of not changing a gene 1 − Pm (Pm probability of
mutation).
Probability of requiring that all o(H) non * genes survive,
(1 − Pm)o(H)
.
Typically the probability of applying the mutation operator, pm 1.
The probability that the mutation disrupt the schema H
Pdisruption (H, mutation) = 1 − (1 − Pm)o(H)
≈ o (H) Pm (2)
After ignoring high terms in the polynomial!!!
13 / 37

34. Remarks about Mutation
Observation about Mutation
Mutation is applied gene by gene.
In order for schema H to survive, all non * genes in the schema much
remain unchanged.
Thus
Probability of not changing a gene 1 − Pm (Pm probability of
mutation).
Probability of requiring that all o(H) non * genes survive,
(1 − Pm)o(H)
.
Typically the probability of applying the mutation operator, pm 1.
The probability that the mutation disrupt the schema H
Pdisruption (H, mutation) = 1 − (1 − Pm)o(H)
≈ o (H) Pm (2)
After ignoring high terms in the polynomial!!!
13 / 37

35. Remarks about Mutation
Observation about Mutation
Mutation is applied gene by gene.
In order for schema H to survive, all non * genes in the schema much
remain unchanged.
Thus
Probability of not changing a gene 1 − Pm (Pm probability of
mutation).
Probability of requiring that all o(H) non * genes survive,
(1 − Pm)o(H)
.
Typically the probability of applying the mutation operator, pm 1.
The probability that the mutation disrupt the schema H
Pdisruption (H, mutation) = 1 − (1 − Pm)o(H)
≈ o (H) Pm (2)
After ignoring high terms in the polynomial!!!
13 / 37

36. Remarks about Mutation
Observation about Mutation
Mutation is applied gene by gene.
In order for schema H to survive, all non * genes in the schema much
remain unchanged.
Thus
Probability of not changing a gene 1 − Pm (Pm probability of
mutation).
Probability of requiring that all o(H) non * genes survive,
(1 − Pm)o(H)
.
Typically the probability of applying the mutation operator, pm 1.
The probability that the mutation disrupt the schema H
Pdisruption (H, mutation) = 1 − (1 − Pm)o(H)
≈ o (H) Pm (2)
After ignoring high terms in the polynomial!!!
13 / 37

37. Remarks about Mutation
Observation about Mutation
Mutation is applied gene by gene.
In order for schema H to survive, all non * genes in the schema much
remain unchanged.
Thus
Probability of not changing a gene 1 − Pm (Pm probability of
mutation).
Probability of requiring that all o(H) non * genes survive,
(1 − Pm)o(H)
.
Typically the probability of applying the mutation operator, pm 1.
The probability that the mutation disrupt the schema H
Pdisruption (H, mutation) = 1 − (1 − Pm)o(H)
≈ o (H) Pm (2)
After ignoring high terms in the polynomial!!!
13 / 37

38. Remarks about Mutation
Observation about Mutation
Mutation is applied gene by gene.
In order for schema H to survive, all non * genes in the schema much
remain unchanged.
Thus
Probability of not changing a gene 1 − Pm (Pm probability of
mutation).
Probability of requiring that all o(H) non * genes survive,
(1 − Pm)o(H)
.
Typically the probability of applying the mutation operator, pm 1.
The probability that the mutation disrupt the schema H
Pdisruption (H, mutation) = 1 − (1 − Pm)o(H)
≈ o (H) Pm (2)
After ignoring high terms in the polynomial!!!
13 / 37

39. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
14 / 37

40. Gene wise Mutation
Lemma 1
Under gene wise mutation (Applied Gene by Gene), the (lower bound)
probability of an order o(H) schema H surviving at generation (No
Disruption) t is,
1 − o (H) Pm (3)
15 / 37

41. Probability of an individual is sampled from schema H
Consider the Following
1 Probability of selection depends on
1 Number of instances of schema H in the population.
2 Average fitness of schema H relative to the average fitness of all
individuals in the population.
Thus, we have the following probability
P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio
16 / 37

42. Probability of an individual is sampled from schema H
Consider the Following
1 Probability of selection depends on
1 Number of instances of schema H in the population.
2 Average fitness of schema H relative to the average fitness of all
individuals in the population.
Thus, we have the following probability
P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio
16 / 37

43. Probability of an individual is sampled from schema H
Consider the Following
1 Probability of selection depends on
1 Number of instances of schema H in the population.
2 Average fitness of schema H relative to the average fitness of all
individuals in the population.
Thus, we have the following probability
P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio
16 / 37

44. Probability of an individual is sampled from schema H
Consider the Following
1 Probability of selection depends on
1 Number of instances of schema H in the population.
2 Average fitness of schema H relative to the average fitness of all
individuals in the population.
Thus, we have the following probability
P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio
16 / 37

45. Then
Finally
P (h ∈ H) =
(Number of individuals
matching schema
H at generation t)
(Population Size)
×
(Mean fitness of
individuals matching
schema H)
(Mean fitness of individuals in the
population)
(4)
17 / 37

46. Then
Finally
P (h ∈ H) =
m (H, t) f (H, t)
Mf (t)
(5)
where M is the population size and m(H, t) is the number of instances of
schema H at generation t.
Lemma 2
Under fitness proportional selection the expected number of instances of
schema H at time t is
E [m (H, t + 1)] = M · P (h ∈ H) =
m (H, t) f (H, t)
f (t)
(6)
18 / 37

47. Then
Finally
P (h ∈ H) =
m (H, t) f (H, t)
Mf (t)
(5)
where M is the population size and m(H, t) is the number of instances of
schema H at generation t.
Lemma 2
Under fitness proportional selection the expected number of instances of
schema H at time t is
E [m (H, t + 1)] = M · P (h ∈ H) =
m (H, t) f (H, t)
f (t)
(6)
18 / 37

48. Why?
Note the following
M independent samples (Same Probability) are taken to create the next
generation of parents
Thus
m (H, t + 1) = Ih1 + Ih2 + ... + IhM
Remark: The indicator random variable of ONE for these samples!!!
Then
E [m (H, t + 1)] = E [Ih1 ] + E [Ih2 ] + ... + E [IhM
]
19 / 37

49. Why?
Note the following
M independent samples (Same Probability) are taken to create the next
generation of parents
Thus
m (H, t + 1) = Ih1 + Ih2 + ... + IhM
Remark: The indicator random variable of ONE for these samples!!!
Then
E [m (H, t + 1)] = E [Ih1 ] + E [Ih2 ] + ... + E [IhM
]
19 / 37

50. Why?
Note the following
M independent samples (Same Probability) are taken to create the next
generation of parents
Thus
m (H, t + 1) = Ih1 + Ih2 + ... + IhM
Remark: The indicator random variable of ONE for these samples!!!
Then
E [m (H, t + 1)] = E [Ih1 ] + E [Ih2 ] + ... + E [IhM
]
19 / 37

51. Finally
But, M samples are taken to create the next generation of parents
E [m (H, t + 1)] = P (h1 ∈ H) + P (h2 ∈ H) + ... + P (hM ∈ H)
Remember the Lemma 5.1 in Cormen’s Book
Finally, because P (h1 ∈ H) = P (h2 ∈ H) = ... = P (hM ∈ H)
E [m (H, t + 1)] = M × P (h ∈ H)
QED!!!
20 / 37

52. Finally
But, M samples are taken to create the next generation of parents
E [m (H, t + 1)] = P (h1 ∈ H) + P (h2 ∈ H) + ... + P (hM ∈ H)
Remember the Lemma 5.1 in Cormen’s Book
Finally, because P (h1 ∈ H) = P (h2 ∈ H) = ... = P (hM ∈ H)
E [m (H, t + 1)] = M × P (h ∈ H)
QED!!!
20 / 37

53. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
21 / 37

54. Search Operators – Single point crossover
Observations
Crossover was the first of two search operators introduced to modify
the distribution of schema in the population.
Holland concentrated on modeling the lower bound alone.
22 / 37

55. Search Operators – Single point crossover
Observations
Crossover was the first of two search operators introduced to modify
the distribution of schema in the population.
Holland concentrated on modeling the lower bound alone.
22 / 37

56. Crossover
Consider the following
Generated Individual h = 1 0 1 | 1 1 0 0
H1 = * 0 1 | * * * 0
H2 = * 0 1 | * * * *
Crossover
Remarks
1 Schema H1 will naturally be broken by the location of the crossover
operator unless the second parent is able to ‘repair’ the disrupted
gene.
2 Schema H2 emerges unaffected and is therefore independent of the
second parent.
3 Thus, Schema with long defining length are more likely to be
disrupted by single point crossover than schema using short
defining lengths.
23 / 37

57. Crossover
Consider the following
Generated Individual h = 1 0 1 | 1 1 0 0
H1 = * 0 1 | * * * 0
H2 = * 0 1 | * * * *
Crossover
Remarks
1 Schema H1 will naturally be broken by the location of the crossover
operator unless the second parent is able to ‘repair’ the disrupted
gene.
2 Schema H2 emerges unaffected and is therefore independent of the
second parent.
3 Thus, Schema with long defining length are more likely to be
disrupted by single point crossover than schema using short
defining lengths.
23 / 37

58. Crossover
Consider the following
Generated Individual h = 1 0 1 | 1 1 0 0
H1 = * 0 1 | * * * 0
H2 = * 0 1 | * * * *
Crossover
Remarks
1 Schema H1 will naturally be broken by the location of the crossover
operator unless the second parent is able to ‘repair’ the disrupted
gene.
2 Schema H2 emerges unaffected and is therefore independent of the
second parent.
3 Thus, Schema with long defining length are more likely to be
disrupted by single point crossover than schema using short
defining lengths.
23 / 37

59. Crossover
Consider the following
Generated Individual h = 1 0 1 | 1 1 0 0
H1 = * 0 1 | * * * 0
H2 = * 0 1 | * * * *
Crossover
Remarks
1 Schema H1 will naturally be broken by the location of the crossover
operator unless the second parent is able to ‘repair’ the disrupted
gene.
2 Schema H2 emerges unaffected and is therefore independent of the
second parent.
3 Thus, Schema with long defining length are more likely to be
disrupted by single point crossover than schema using short
defining lengths.
23 / 37

60. Now, we have
Lemma 3
Under single point crossover, the (lower bound) probability of schema H
surviving at generation t is,
Pcrossover (H survive) =1 − Pcrossover (H does not survive)
=1 − pc
δ(H)
l − 1
Pdiff (H, t)
Where
Pdiff (H, t) is the probability that the second parent does not
match schema H.
pc is the a priori selected threshold of applying crossover.
24 / 37

61. Now, we have
Lemma 3
Under single point crossover, the (lower bound) probability of schema H
surviving at generation t is,
Pcrossover (H survive) =1 − Pcrossover (H does not survive)
=1 − pc
δ(H)
l − 1
Pdiff (H, t)
Where
Pdiff (H, t) is the probability that the second parent does not
match schema H.
pc is the a priori selected threshold of applying crossover.
24 / 37

62. Now, we have
Lemma 3
Under single point crossover, the (lower bound) probability of schema H
surviving at generation t is,
Pcrossover (H survive) =1 − Pcrossover (H does not survive)
=1 − pc
δ(H)
l − 1
Pdiff (H, t)
Where
Pdiff (H, t) is the probability that the second parent does not
match schema H.
pc is the a priori selected threshold of applying crossover.
24 / 37

63. How?
We can see the following
Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t)
After all
Pc is used to decide if the crossover will happen.
The second parent could come from the same schema, and yes!!! We
do not have a disruption!!!
Then
Pcrossover (H does not survive) = Pc ×
δ(H)
l − 1
× Pdiff (H, t)
25 / 37

64. How?
We can see the following
Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t)
After all
Pc is used to decide if the crossover will happen.
The second parent could come from the same schema, and yes!!! We
do not have a disruption!!!
Then
Pcrossover (H does not survive) = Pc ×
δ(H)
l − 1
× Pdiff (H, t)
25 / 37

65. How?
We can see the following
Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t)
After all
Pc is used to decide if the crossover will happen.
The second parent could come from the same schema, and yes!!! We
do not have a disruption!!!
Then
Pcrossover (H does not survive) = Pc ×
δ(H)
l − 1
× Pdiff (H, t)
25 / 37

66. How?
We can see the following
Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t)
After all
Pc is used to decide if the crossover will happen.
The second parent could come from the same schema, and yes!!! We
do not have a disruption!!!
Then
Pcrossover (H does not survive) = Pc ×
δ(H)
l − 1
× Pdiff (H, t)
25 / 37

67. In addition
Worst case lower bound
Pdiff (H, t) = 1 (7)
26 / 37

68. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
27 / 37

69. The Schema Theorem
The Schema Theorem
The expected number of schema H at generation t + 1 when using a
canonical GA with proportional selection, single point crossover and gene
wise mutation (where the latter are applied at rates pc and Pm) is,
E [m (H, t + 1)] ≥
m (H, t) f (H, t)
f (t)
1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm
(8)
28 / 37

70. Proof
We use the following quantities
Pcrossover (H survive) = 1 − pc
δ(H)
l−1 Pdiff (H, t) ≤ 1
Pno−disruption (H, mutation) = 1 − o (H) Pm ≤ 1
Then, we have that
E [m (H, t + 1)] =M × P (h ∈ H)
=M
m (H, t) f (H, t)
Mf (t)
=
m (H, t) f (H, t)
f (t)
≥
m (H, t) f (H, t)
f (t)
× 1 − pc
δ(H)
l − 1
Pdiff (H, t) × [1 − o (H) Pm]
29 / 37

71. Proof
We use the following quantities
Pcrossover (H survive) = 1 − pc
δ(H)
l−1 Pdiff (H, t) ≤ 1
Pno−disruption (H, mutation) = 1 − o (H) Pm ≤ 1
Then, we have that
E [m (H, t + 1)] =M × P (h ∈ H)
=M
m (H, t) f (H, t)
Mf (t)
=
m (H, t) f (H, t)
f (t)
≥
m (H, t) f (H, t)
f (t)
× 1 − pc
δ(H)
l − 1
Pdiff (H, t) × [1 − o (H) Pm]
29 / 37

72. Proof
We use the following quantities
Pcrossover (H survive) = 1 − pc
δ(H)
l−1 Pdiff (H, t) ≤ 1
Pno−disruption (H, mutation) = 1 − o (H) Pm ≤ 1
Then, we have that
E [m (H, t + 1)] =M × P (h ∈ H)
=M
m (H, t) f (H, t)
Mf (t)
=
m (H, t) f (H, t)
f (t)
≥
m (H, t) f (H, t)
f (t)
× 1 − pc
δ(H)
l − 1
Pdiff (H, t) × [1 − o (H) Pm]
29 / 37

73. Thus
We have the following
E [m (H, t + 1)] ≥
m (H, t) f (H, t)
f (t)
1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm + ...
pc
δ(H)
l − 1
Pdiff (H, t)o (H) Pm
≥
m (H, t) f (H, t)
f (t)
1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm
The las inequality is possible because pc
δ(H)
l−1 Pdiff (H, t)o (H) Pm ≥ 0
30 / 37

74. Remarks
Observations
The theorem is described in terms of expectation, thus strictly
speaking is only true for the case of a population with an infinite
number of members.
What about a finite population?
In the case of finite population sizes the significance of population drift
plays an increasingly important role.
31 / 37

75. Remarks
Observations
The theorem is described in terms of expectation, thus strictly
speaking is only true for the case of a population with an infinite
number of members.
What about a finite population?
In the case of finite population sizes the significance of population drift
plays an increasingly important role.
31 / 37

76. Remarks
Observations
The theorem is described in terms of expectation, thus strictly
speaking is only true for the case of a population with an infinite
number of members.
What about a finite population?
In the case of finite population sizes the significance of population drift
plays an increasingly important role.
31 / 37

77. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
32 / 37

78. More General Version
More General Version
E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9)
Where
α(H, t)is the “selection coefficient”
β(H, t) is the “transcription error.”
This allows to say that H survives if
α(H, t) ≥ 1 − β (H, t) or
m (H, t) f (H, t)
f (t)
≥ 1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm
33 / 37

79. More General Version
More General Version
E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9)
Where
α(H, t)is the “selection coefficient”
β(H, t) is the “transcription error.”
This allows to say that H survives if
α(H, t) ≥ 1 − β (H, t) or
m (H, t) f (H, t)
f (t)
≥ 1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm
33 / 37

80. More General Version
More General Version
E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9)
Where
α(H, t)is the “selection coefficient”
β(H, t) is the “transcription error.”
This allows to say that H survives if
α(H, t) ≥ 1 − β (H, t) or
m (H, t) f (H, t)
f (t)
≥ 1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm
33 / 37

81. More General Version
More General Version
E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9)
Where
α(H, t)is the “selection coefficient”
β(H, t) is the “transcription error.”
This allows to say that H survives if
α(H, t) ≥ 1 − β (H, t) or
m (H, t) f (H, t)
f (t)
≥ 1 − pc
δ(H)
l − 1
Pdiff (H, t) − o (H) Pm
33 / 37

82. Observation
Observation
This is the basis for the observation that short (defining length), low order
schema of above average population fitness will be favored by canonical
GAs, or the Building Block Hypothesis.
34 / 37

83. Outline
1 Introduction
Schema Definition
Properties of Schemas
2 Probability of a Schema
Probability of an individual is in schema H
Surviving Under Gene wise Mutation
Surviving Under Single Point Crossover
The Schema Theorem
A More General Version
Problems with the Schema Theorem
35 / 37

84. Problems
Problem 1
Only the worst-case scenario is considered.
No positive effects of the search operators are considered.
This has lead to the development of Exact Schema Theorems.
Problem 2
The theorem concentrates on the number of schema surviving not
which schema survive.
Such considerations have been addressed by the utilization of Markov
chains to provide models of behavior associated with specific
individuals in the population.
36 / 37

85. Problems
Problem 1
Only the worst-case scenario is considered.
No positive effects of the search operators are considered.
This has lead to the development of Exact Schema Theorems.
Problem 2
The theorem concentrates on the number of schema surviving not
which schema survive.
Such considerations have been addressed by the utilization of Markov
chains to provide models of behavior associated with specific
individuals in the population.
36 / 37

86. Problems
Problem 1
Only the worst-case scenario is considered.
No positive effects of the search operators are considered.
This has lead to the development of Exact Schema Theorems.
Problem 2
The theorem concentrates on the number of schema surviving not
which schema survive.
Such considerations have been addressed by the utilization of Markov
chains to provide models of behavior associated with specific
individuals in the population.
36 / 37

87. Problems
Problem 1
Only the worst-case scenario is considered.
No positive effects of the search operators are considered.
This has lead to the development of Exact Schema Theorems.
Problem 2
The theorem concentrates on the number of schema surviving not
which schema survive.
Such considerations have been addressed by the utilization of Markov
chains to provide models of behavior associated with specific
individuals in the population.
36 / 37

88. Problems
Problem 1
Only the worst-case scenario is considered.
No positive effects of the search operators are considered.
This has lead to the development of Exact Schema Theorems.
Problem 2
The theorem concentrates on the number of schema surviving not
which schema survive.
Such considerations have been addressed by the utilization of Markov
chains to provide models of behavior associated with specific
individuals in the population.
36 / 37

89. Problems
Problem 1
Only the worst-case scenario is considered.
No positive effects of the search operators are considered.
This has lead to the development of Exact Schema Theorems.
Problem 2
The theorem concentrates on the number of schema surviving not
which schema survive.
Such considerations have been addressed by the utilization of Markov
chains to provide models of behavior associated with specific
individuals in the population.
36 / 37

90. Problems
Problem 3
Claims of “exponential increases” in fit schema i.e., if the expectation
operator of Schema Theorem is ignored and the effects of crossover
and mutation discounted, the following result was popularized by
Goldberg,
m(H, t + 1)≥(1 + c)m(H, t)
where c is the constant by which fit schema are always fitter than the
population average.
PROBLEM!!!
Unfortunately, this is rather misleading as the average population
fitness will tend to increase with t,
thus population and fitness of remaining schema will tend to converge
with increasing ‘time’.
37 / 37

91. Problems
Problem 3
Claims of “exponential increases” in fit schema i.e., if the expectation
operator of Schema Theorem is ignored and the effects of crossover
and mutation discounted, the following result was popularized by
Goldberg,
m(H, t + 1)≥(1 + c)m(H, t)
where c is the constant by which fit schema are always fitter than the
population average.
PROBLEM!!!
Unfortunately, this is rather misleading as the average population
fitness will tend to increase with t,
thus population and fitness of remaining schema will tend to converge
with increasing ‘time’.
37 / 37