Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

07.2 Holland's Genetic Algorithms Schema Theorem

3,983 views

Published on

One of the fewest Evolutionary algorithms with proof about the Expected number of parents for a certain Schema. The slides have been updated with a better proof, however, the proof still have some problems... I seriously believe that we need a topology stochastic process to really understand what is going on in Genetic Algorithms. This quite tough because of mixing of topology and probability to define a realistic model of populations in Genetic Algorithms.

Published in: Engineering
  • Sex in your area is here: ♥♥♥ http://bit.ly/39sFWPG ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating for everyone is here: ❤❤❤ http://bit.ly/39sFWPG ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://shorturl.at/mzUV6 } ......................................................................................................................... Download Full EPUB Ebook here { http://shorturl.at/mzUV6 } ......................................................................................................................... Download Full doc Ebook here { http://shorturl.at/mzUV6 } ......................................................................................................................... Download PDF EBOOK here { http://shorturl.at/mzUV6 } ......................................................................................................................... Download EPUB Ebook here { http://shorturl.at/mzUV6 } ......................................................................................................................... Download doc Ebook here { http://shorturl.at/mzUV6 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

07.2 Holland's Genetic Algorithms Schema Theorem

  1. 1. Artificial Intelligence Holland’s GA Schema Theorem Andres Mendez-Vazquez April 7, 2015 1 / 37
  2. 2. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 2 / 37
  3. 3. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 3 / 37
  4. 4. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  5. 5. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  6. 6. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  7. 7. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  8. 8. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  9. 9. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  10. 10. Introduction Consider the Canonical GA Binary alphabet. Fixed length individuals of equal length, l. Fitness Proportional Selection. Single Point Crossover (1X). Gene wise mutation i.e. mutate gene by gene. Definition 1 - Schema H A schema H is a template that identifies a subset of strings with similarities at certain string positions. Schemata are a special case of a natural open set of a product topology. 4 / 37
  11. 11. Introduction Definition 2 If A denotes the alphabet of genes, then A ∪ ∗ is the schema alphabet, where * is the ‘wild card’ symbol matching any gene value. Example: For A ∈ {0, 1, ∗} where ∗ ∈ {0, 1}. 5 / 37
  12. 12. Example The Schema H = [0 1 ∗ 1 *] generates the following individuals 0 1 * 1 * 0 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 Not all schemas say the same Schema [ 1 ∗ ∗ ∗ ∗ ∗ ∗] has less information than [ 0 1 ∗ ∗ 1 1 0]. It is more!!! [ 1 ∗ ∗ ∗ ∗ ∗ 0] span the entire length of an individual, but [ 1 ∗ 1 ∗ ∗ ∗ ∗] does not. 6 / 37
  13. 13. Example The Schema H = [0 1 ∗ 1 *] generates the following individuals 0 1 * 1 * 0 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 Not all schemas say the same Schema [ 1 ∗ ∗ ∗ ∗ ∗ ∗] has less information than [ 0 1 ∗ ∗ 1 1 0]. It is more!!! [ 1 ∗ ∗ ∗ ∗ ∗ 0] span the entire length of an individual, but [ 1 ∗ 1 ∗ ∗ ∗ ∗] does not. 6 / 37
  14. 14. Example The Schema H = [0 1 ∗ 1 *] generates the following individuals 0 1 * 1 * 0 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 Not all schemas say the same Schema [ 1 ∗ ∗ ∗ ∗ ∗ ∗] has less information than [ 0 1 ∗ ∗ 1 1 0]. It is more!!! [ 1 ∗ ∗ ∗ ∗ ∗ 0] span the entire length of an individual, but [ 1 ∗ 1 ∗ ∗ ∗ ∗] does not. 6 / 37
  15. 15. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 7 / 37
  16. 16. Schema Order and Length Definition 3 - Schema Order o (H) Schema order, o, is the number of non “*” genes in schema H. Example: o(***11*1)=3. Definition 3 – Schema Defining Length, δ (H). Schema Defining Length, δ(H), is the distance between first and last non “*” gene in schema H. Example: δ(***11*1)=7-4=3. Notes Given an alphabet A with|A| = k, then there are (k + 1)l possible schemas of length l. 8 / 37
  17. 17. Schema Order and Length Definition 3 - Schema Order o (H) Schema order, o, is the number of non “*” genes in schema H. Example: o(***11*1)=3. Definition 3 – Schema Defining Length, δ (H). Schema Defining Length, δ(H), is the distance between first and last non “*” gene in schema H. Example: δ(***11*1)=7-4=3. Notes Given an alphabet A with|A| = k, then there are (k + 1)l possible schemas of length l. 8 / 37
  18. 18. Schema Order and Length Definition 3 - Schema Order o (H) Schema order, o, is the number of non “*” genes in schema H. Example: o(***11*1)=3. Definition 3 – Schema Defining Length, δ (H). Schema Defining Length, δ(H), is the distance between first and last non “*” gene in schema H. Example: δ(***11*1)=7-4=3. Notes Given an alphabet A with|A| = k, then there are (k + 1)l possible schemas of length l. 8 / 37
  19. 19. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 9 / 37
  20. 20. Probabilities of belonging to a Schema H What do we want? The probability that individual h is from schema H: P (h ∈ H) We need the following probabilities Pdistruption(H, 1X) = probability of schema being disrupted due to crossover. Pdisruption (H, mutation) =probability of schema being disrupted due to mutation Pcrossover (H survive) 10 / 37
  21. 21. Probabilities of belonging to a Schema H What do we want? The probability that individual h is from schema H: P (h ∈ H) We need the following probabilities Pdistruption(H, 1X) = probability of schema being disrupted due to crossover. Pdisruption (H, mutation) =probability of schema being disrupted due to mutation Pcrossover (H survive) 10 / 37
  22. 22. Probabilities of belonging to a Schema H What do we want? The probability that individual h is from schema H: P (h ∈ H) We need the following probabilities Pdistruption(H, 1X) = probability of schema being disrupted due to crossover. Pdisruption (H, mutation) =probability of schema being disrupted due to mutation Pcrossover (H survive) 10 / 37
  23. 23. Probability of Disruption Consider now The CGA using fitness proportionate parent selection on-point crossover (1X) bitwise mutation with probability Pm Genotypes of length l The Schema could be disrupted if the cross over falls between the ends Pdistruption(H, 1X) = δ(H) (l − 1) (1) 0 1 0 0 1 0 11 / 37
  24. 24. Probability of Disruption Consider now The CGA using fitness proportionate parent selection on-point crossover (1X) bitwise mutation with probability Pm Genotypes of length l The Schema could be disrupted if the cross over falls between the ends Pdistruption(H, 1X) = δ(H) (l − 1) (1) 0 1 0 0 1 0 11 / 37
  25. 25. Probability of Disruption Consider now The CGA using fitness proportionate parent selection on-point crossover (1X) bitwise mutation with probability Pm Genotypes of length l The Schema could be disrupted if the cross over falls between the ends Pdistruption(H, 1X) = δ(H) (l − 1) (1) 0 1 0 0 1 0 11 / 37
  26. 26. Probability of Disruption Consider now The CGA using fitness proportionate parent selection on-point crossover (1X) bitwise mutation with probability Pm Genotypes of length l The Schema could be disrupted if the cross over falls between the ends Pdistruption(H, 1X) = δ(H) (l − 1) (1) 0 1 0 0 1 0 11 / 37
  27. 27. Probability of Disruption Consider now The CGA using fitness proportionate parent selection on-point crossover (1X) bitwise mutation with probability Pm Genotypes of length l The Schema could be disrupted if the cross over falls between the ends Pdistruption(H, 1X) = δ(H) (l − 1) (1) 0 1 0 0 1 0 11 / 37
  28. 28. Probability of Disruption Consider now The CGA using fitness proportionate parent selection on-point crossover (1X) bitwise mutation with probability Pm Genotypes of length l The Schema could be disrupted if the cross over falls between the ends Pdistruption(H, 1X) = δ(H) (l − 1) (1) 0 1 0 0 1 0 11 / 37
  29. 29. Why? Given that you have δ(H) = the distance between first and last non “*” last position in Genotype - first position in Genotype = l − 1 Case I δ(H) = 1, when the positions of the non “*” are next to each other Case II δ(H) = l − 1, when the positions of the non “*” are in the extremes 12 / 37
  30. 30. Why? Given that you have δ(H) = the distance between first and last non “*” last position in Genotype - first position in Genotype = l − 1 Case I δ(H) = 1, when the positions of the non “*” are next to each other Case II δ(H) = l − 1, when the positions of the non “*” are in the extremes 12 / 37
  31. 31. Why? Given that you have δ(H) = the distance between first and last non “*” last position in Genotype - first position in Genotype = l − 1 Case I δ(H) = 1, when the positions of the non “*” are next to each other Case II δ(H) = l − 1, when the positions of the non “*” are in the extremes 12 / 37
  32. 32. Why? Given that you have δ(H) = the distance between first and last non “*” last position in Genotype - first position in Genotype = l − 1 Case I δ(H) = 1, when the positions of the non “*” are next to each other Case II δ(H) = l − 1, when the positions of the non “*” are in the extremes 12 / 37
  33. 33. Remarks about Mutation Observation about Mutation Mutation is applied gene by gene. In order for schema H to survive, all non * genes in the schema much remain unchanged. Thus Probability of not changing a gene 1 − Pm (Pm probability of mutation). Probability of requiring that all o(H) non * genes survive, (1 − Pm)o(H) . Typically the probability of applying the mutation operator, pm 1. The probability that the mutation disrupt the schema H Pdisruption (H, mutation) = 1 − (1 − Pm)o(H) ≈ o (H) Pm (2) After ignoring high terms in the polynomial!!! 13 / 37
  34. 34. Remarks about Mutation Observation about Mutation Mutation is applied gene by gene. In order for schema H to survive, all non * genes in the schema much remain unchanged. Thus Probability of not changing a gene 1 − Pm (Pm probability of mutation). Probability of requiring that all o(H) non * genes survive, (1 − Pm)o(H) . Typically the probability of applying the mutation operator, pm 1. The probability that the mutation disrupt the schema H Pdisruption (H, mutation) = 1 − (1 − Pm)o(H) ≈ o (H) Pm (2) After ignoring high terms in the polynomial!!! 13 / 37
  35. 35. Remarks about Mutation Observation about Mutation Mutation is applied gene by gene. In order for schema H to survive, all non * genes in the schema much remain unchanged. Thus Probability of not changing a gene 1 − Pm (Pm probability of mutation). Probability of requiring that all o(H) non * genes survive, (1 − Pm)o(H) . Typically the probability of applying the mutation operator, pm 1. The probability that the mutation disrupt the schema H Pdisruption (H, mutation) = 1 − (1 − Pm)o(H) ≈ o (H) Pm (2) After ignoring high terms in the polynomial!!! 13 / 37
  36. 36. Remarks about Mutation Observation about Mutation Mutation is applied gene by gene. In order for schema H to survive, all non * genes in the schema much remain unchanged. Thus Probability of not changing a gene 1 − Pm (Pm probability of mutation). Probability of requiring that all o(H) non * genes survive, (1 − Pm)o(H) . Typically the probability of applying the mutation operator, pm 1. The probability that the mutation disrupt the schema H Pdisruption (H, mutation) = 1 − (1 − Pm)o(H) ≈ o (H) Pm (2) After ignoring high terms in the polynomial!!! 13 / 37
  37. 37. Remarks about Mutation Observation about Mutation Mutation is applied gene by gene. In order for schema H to survive, all non * genes in the schema much remain unchanged. Thus Probability of not changing a gene 1 − Pm (Pm probability of mutation). Probability of requiring that all o(H) non * genes survive, (1 − Pm)o(H) . Typically the probability of applying the mutation operator, pm 1. The probability that the mutation disrupt the schema H Pdisruption (H, mutation) = 1 − (1 − Pm)o(H) ≈ o (H) Pm (2) After ignoring high terms in the polynomial!!! 13 / 37
  38. 38. Remarks about Mutation Observation about Mutation Mutation is applied gene by gene. In order for schema H to survive, all non * genes in the schema much remain unchanged. Thus Probability of not changing a gene 1 − Pm (Pm probability of mutation). Probability of requiring that all o(H) non * genes survive, (1 − Pm)o(H) . Typically the probability of applying the mutation operator, pm 1. The probability that the mutation disrupt the schema H Pdisruption (H, mutation) = 1 − (1 − Pm)o(H) ≈ o (H) Pm (2) After ignoring high terms in the polynomial!!! 13 / 37
  39. 39. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 14 / 37
  40. 40. Gene wise Mutation Lemma 1 Under gene wise mutation (Applied Gene by Gene), the (lower bound) probability of an order o(H) schema H surviving at generation (No Disruption) t is, 1 − o (H) Pm (3) 15 / 37
  41. 41. Probability of an individual is sampled from schema H Consider the Following 1 Probability of selection depends on 1 Number of instances of schema H in the population. 2 Average fitness of schema H relative to the average fitness of all individuals in the population. Thus, we have the following probability P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio 16 / 37
  42. 42. Probability of an individual is sampled from schema H Consider the Following 1 Probability of selection depends on 1 Number of instances of schema H in the population. 2 Average fitness of schema H relative to the average fitness of all individuals in the population. Thus, we have the following probability P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio 16 / 37
  43. 43. Probability of an individual is sampled from schema H Consider the Following 1 Probability of selection depends on 1 Number of instances of schema H in the population. 2 Average fitness of schema H relative to the average fitness of all individuals in the population. Thus, we have the following probability P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio 16 / 37
  44. 44. Probability of an individual is sampled from schema H Consider the Following 1 Probability of selection depends on 1 Number of instances of schema H in the population. 2 Average fitness of schema H relative to the average fitness of all individuals in the population. Thus, we have the following probability P (h ∈ H) = PUniform (h in Population) × Mean Fitness Ratio 16 / 37
  45. 45. Then Finally P (h ∈ H) = (Number of individuals matching schema H at generation t) (Population Size) × (Mean fitness of individuals matching schema H) (Mean fitness of individuals in the population) (4) 17 / 37
  46. 46. Then Finally P (h ∈ H) = m (H, t) f (H, t) Mf (t) (5) where M is the population size and m(H, t) is the number of instances of schema H at generation t. Lemma 2 Under fitness proportional selection the expected number of instances of schema H at time t is E [m (H, t + 1)] = M · P (h ∈ H) = m (H, t) f (H, t) f (t) (6) 18 / 37
  47. 47. Then Finally P (h ∈ H) = m (H, t) f (H, t) Mf (t) (5) where M is the population size and m(H, t) is the number of instances of schema H at generation t. Lemma 2 Under fitness proportional selection the expected number of instances of schema H at time t is E [m (H, t + 1)] = M · P (h ∈ H) = m (H, t) f (H, t) f (t) (6) 18 / 37
  48. 48. Why? Note the following M independent samples (Same Probability) are taken to create the next generation of parents Thus m (H, t + 1) = Ih1 + Ih2 + ... + IhM Remark: The indicator random variable of ONE for these samples!!! Then E [m (H, t + 1)] = E [Ih1 ] + E [Ih2 ] + ... + E [IhM ] 19 / 37
  49. 49. Why? Note the following M independent samples (Same Probability) are taken to create the next generation of parents Thus m (H, t + 1) = Ih1 + Ih2 + ... + IhM Remark: The indicator random variable of ONE for these samples!!! Then E [m (H, t + 1)] = E [Ih1 ] + E [Ih2 ] + ... + E [IhM ] 19 / 37
  50. 50. Why? Note the following M independent samples (Same Probability) are taken to create the next generation of parents Thus m (H, t + 1) = Ih1 + Ih2 + ... + IhM Remark: The indicator random variable of ONE for these samples!!! Then E [m (H, t + 1)] = E [Ih1 ] + E [Ih2 ] + ... + E [IhM ] 19 / 37
  51. 51. Finally But, M samples are taken to create the next generation of parents E [m (H, t + 1)] = P (h1 ∈ H) + P (h2 ∈ H) + ... + P (hM ∈ H) Remember the Lemma 5.1 in Cormen’s Book Finally, because P (h1 ∈ H) = P (h2 ∈ H) = ... = P (hM ∈ H) E [m (H, t + 1)] = M × P (h ∈ H) QED!!! 20 / 37
  52. 52. Finally But, M samples are taken to create the next generation of parents E [m (H, t + 1)] = P (h1 ∈ H) + P (h2 ∈ H) + ... + P (hM ∈ H) Remember the Lemma 5.1 in Cormen’s Book Finally, because P (h1 ∈ H) = P (h2 ∈ H) = ... = P (hM ∈ H) E [m (H, t + 1)] = M × P (h ∈ H) QED!!! 20 / 37
  53. 53. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 21 / 37
  54. 54. Search Operators – Single point crossover Observations Crossover was the first of two search operators introduced to modify the distribution of schema in the population. Holland concentrated on modeling the lower bound alone. 22 / 37
  55. 55. Search Operators – Single point crossover Observations Crossover was the first of two search operators introduced to modify the distribution of schema in the population. Holland concentrated on modeling the lower bound alone. 22 / 37
  56. 56. Crossover Consider the following Generated Individual h = 1 0 1 | 1 1 0 0 H1 = * 0 1 | * * * 0 H2 = * 0 1 | * * * * Crossover Remarks 1 Schema H1 will naturally be broken by the location of the crossover operator unless the second parent is able to ‘repair’ the disrupted gene. 2 Schema H2 emerges unaffected and is therefore independent of the second parent. 3 Thus, Schema with long defining length are more likely to be disrupted by single point crossover than schema using short defining lengths. 23 / 37
  57. 57. Crossover Consider the following Generated Individual h = 1 0 1 | 1 1 0 0 H1 = * 0 1 | * * * 0 H2 = * 0 1 | * * * * Crossover Remarks 1 Schema H1 will naturally be broken by the location of the crossover operator unless the second parent is able to ‘repair’ the disrupted gene. 2 Schema H2 emerges unaffected and is therefore independent of the second parent. 3 Thus, Schema with long defining length are more likely to be disrupted by single point crossover than schema using short defining lengths. 23 / 37
  58. 58. Crossover Consider the following Generated Individual h = 1 0 1 | 1 1 0 0 H1 = * 0 1 | * * * 0 H2 = * 0 1 | * * * * Crossover Remarks 1 Schema H1 will naturally be broken by the location of the crossover operator unless the second parent is able to ‘repair’ the disrupted gene. 2 Schema H2 emerges unaffected and is therefore independent of the second parent. 3 Thus, Schema with long defining length are more likely to be disrupted by single point crossover than schema using short defining lengths. 23 / 37
  59. 59. Crossover Consider the following Generated Individual h = 1 0 1 | 1 1 0 0 H1 = * 0 1 | * * * 0 H2 = * 0 1 | * * * * Crossover Remarks 1 Schema H1 will naturally be broken by the location of the crossover operator unless the second parent is able to ‘repair’ the disrupted gene. 2 Schema H2 emerges unaffected and is therefore independent of the second parent. 3 Thus, Schema with long defining length are more likely to be disrupted by single point crossover than schema using short defining lengths. 23 / 37
  60. 60. Now, we have Lemma 3 Under single point crossover, the (lower bound) probability of schema H surviving at generation t is, Pcrossover (H survive) =1 − Pcrossover (H does not survive) =1 − pc δ(H) l − 1 Pdiff (H, t) Where Pdiff (H, t) is the probability that the second parent does not match schema H. pc is the a priori selected threshold of applying crossover. 24 / 37
  61. 61. Now, we have Lemma 3 Under single point crossover, the (lower bound) probability of schema H surviving at generation t is, Pcrossover (H survive) =1 − Pcrossover (H does not survive) =1 − pc δ(H) l − 1 Pdiff (H, t) Where Pdiff (H, t) is the probability that the second parent does not match schema H. pc is the a priori selected threshold of applying crossover. 24 / 37
  62. 62. Now, we have Lemma 3 Under single point crossover, the (lower bound) probability of schema H surviving at generation t is, Pcrossover (H survive) =1 − Pcrossover (H does not survive) =1 − pc δ(H) l − 1 Pdiff (H, t) Where Pdiff (H, t) is the probability that the second parent does not match schema H. pc is the a priori selected threshold of applying crossover. 24 / 37
  63. 63. How? We can see the following Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t) After all Pc is used to decide if the crossover will happen. The second parent could come from the same schema, and yes!!! We do not have a disruption!!! Then Pcrossover (H does not survive) = Pc × δ(H) l − 1 × Pdiff (H, t) 25 / 37
  64. 64. How? We can see the following Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t) After all Pc is used to decide if the crossover will happen. The second parent could come from the same schema, and yes!!! We do not have a disruption!!! Then Pcrossover (H does not survive) = Pc × δ(H) l − 1 × Pdiff (H, t) 25 / 37
  65. 65. How? We can see the following Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t) After all Pc is used to decide if the crossover will happen. The second parent could come from the same schema, and yes!!! We do not have a disruption!!! Then Pcrossover (H does not survive) = Pc × δ(H) l − 1 × Pdiff (H, t) 25 / 37
  66. 66. How? We can see the following Pcrossover (H does not survive) = Pc × Pdistruption(H, 1X) × Pdiff (H, t) After all Pc is used to decide if the crossover will happen. The second parent could come from the same schema, and yes!!! We do not have a disruption!!! Then Pcrossover (H does not survive) = Pc × δ(H) l − 1 × Pdiff (H, t) 25 / 37
  67. 67. In addition Worst case lower bound Pdiff (H, t) = 1 (7) 26 / 37
  68. 68. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 27 / 37
  69. 69. The Schema Theorem The Schema Theorem The expected number of schema H at generation t + 1 when using a canonical GA with proportional selection, single point crossover and gene wise mutation (where the latter are applied at rates pc and Pm) is, E [m (H, t + 1)] ≥ m (H, t) f (H, t) f (t) 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm (8) 28 / 37
  70. 70. Proof We use the following quantities Pcrossover (H survive) = 1 − pc δ(H) l−1 Pdiff (H, t) ≤ 1 Pno−disruption (H, mutation) = 1 − o (H) Pm ≤ 1 Then, we have that E [m (H, t + 1)] =M × P (h ∈ H) =M m (H, t) f (H, t) Mf (t) = m (H, t) f (H, t) f (t) ≥ m (H, t) f (H, t) f (t) × 1 − pc δ(H) l − 1 Pdiff (H, t) × [1 − o (H) Pm] 29 / 37
  71. 71. Proof We use the following quantities Pcrossover (H survive) = 1 − pc δ(H) l−1 Pdiff (H, t) ≤ 1 Pno−disruption (H, mutation) = 1 − o (H) Pm ≤ 1 Then, we have that E [m (H, t + 1)] =M × P (h ∈ H) =M m (H, t) f (H, t) Mf (t) = m (H, t) f (H, t) f (t) ≥ m (H, t) f (H, t) f (t) × 1 − pc δ(H) l − 1 Pdiff (H, t) × [1 − o (H) Pm] 29 / 37
  72. 72. Proof We use the following quantities Pcrossover (H survive) = 1 − pc δ(H) l−1 Pdiff (H, t) ≤ 1 Pno−disruption (H, mutation) = 1 − o (H) Pm ≤ 1 Then, we have that E [m (H, t + 1)] =M × P (h ∈ H) =M m (H, t) f (H, t) Mf (t) = m (H, t) f (H, t) f (t) ≥ m (H, t) f (H, t) f (t) × 1 − pc δ(H) l − 1 Pdiff (H, t) × [1 − o (H) Pm] 29 / 37
  73. 73. Thus We have the following E [m (H, t + 1)] ≥ m (H, t) f (H, t) f (t) 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm + ... pc δ(H) l − 1 Pdiff (H, t)o (H) Pm ≥ m (H, t) f (H, t) f (t) 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm The las inequality is possible because pc δ(H) l−1 Pdiff (H, t)o (H) Pm ≥ 0 30 / 37
  74. 74. Remarks Observations The theorem is described in terms of expectation, thus strictly speaking is only true for the case of a population with an infinite number of members. What about a finite population? In the case of finite population sizes the significance of population drift plays an increasingly important role. 31 / 37
  75. 75. Remarks Observations The theorem is described in terms of expectation, thus strictly speaking is only true for the case of a population with an infinite number of members. What about a finite population? In the case of finite population sizes the significance of population drift plays an increasingly important role. 31 / 37
  76. 76. Remarks Observations The theorem is described in terms of expectation, thus strictly speaking is only true for the case of a population with an infinite number of members. What about a finite population? In the case of finite population sizes the significance of population drift plays an increasingly important role. 31 / 37
  77. 77. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 32 / 37
  78. 78. More General Version More General Version E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9) Where α(H, t)is the “selection coefficient” β(H, t) is the “transcription error.” This allows to say that H survives if α(H, t) ≥ 1 − β (H, t) or m (H, t) f (H, t) f (t) ≥ 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm 33 / 37
  79. 79. More General Version More General Version E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9) Where α(H, t)is the “selection coefficient” β(H, t) is the “transcription error.” This allows to say that H survives if α(H, t) ≥ 1 − β (H, t) or m (H, t) f (H, t) f (t) ≥ 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm 33 / 37
  80. 80. More General Version More General Version E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9) Where α(H, t)is the “selection coefficient” β(H, t) is the “transcription error.” This allows to say that H survives if α(H, t) ≥ 1 − β (H, t) or m (H, t) f (H, t) f (t) ≥ 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm 33 / 37
  81. 81. More General Version More General Version E [m (H, t + 1)] ≥ m (H, t) α (H, t) {1 − β(H, t)} (9) Where α(H, t)is the “selection coefficient” β(H, t) is the “transcription error.” This allows to say that H survives if α(H, t) ≥ 1 − β (H, t) or m (H, t) f (H, t) f (t) ≥ 1 − pc δ(H) l − 1 Pdiff (H, t) − o (H) Pm 33 / 37
  82. 82. Observation Observation This is the basis for the observation that short (defining length), low order schema of above average population fitness will be favored by canonical GAs, or the Building Block Hypothesis. 34 / 37
  83. 83. Outline 1 Introduction Schema Definition Properties of Schemas 2 Probability of a Schema Probability of an individual is in schema H Surviving Under Gene wise Mutation Surviving Under Single Point Crossover The Schema Theorem A More General Version Problems with the Schema Theorem 35 / 37
  84. 84. Problems Problem 1 Only the worst-case scenario is considered. No positive effects of the search operators are considered. This has lead to the development of Exact Schema Theorems. Problem 2 The theorem concentrates on the number of schema surviving not which schema survive. Such considerations have been addressed by the utilization of Markov chains to provide models of behavior associated with specific individuals in the population. 36 / 37
  85. 85. Problems Problem 1 Only the worst-case scenario is considered. No positive effects of the search operators are considered. This has lead to the development of Exact Schema Theorems. Problem 2 The theorem concentrates on the number of schema surviving not which schema survive. Such considerations have been addressed by the utilization of Markov chains to provide models of behavior associated with specific individuals in the population. 36 / 37
  86. 86. Problems Problem 1 Only the worst-case scenario is considered. No positive effects of the search operators are considered. This has lead to the development of Exact Schema Theorems. Problem 2 The theorem concentrates on the number of schema surviving not which schema survive. Such considerations have been addressed by the utilization of Markov chains to provide models of behavior associated with specific individuals in the population. 36 / 37
  87. 87. Problems Problem 1 Only the worst-case scenario is considered. No positive effects of the search operators are considered. This has lead to the development of Exact Schema Theorems. Problem 2 The theorem concentrates on the number of schema surviving not which schema survive. Such considerations have been addressed by the utilization of Markov chains to provide models of behavior associated with specific individuals in the population. 36 / 37
  88. 88. Problems Problem 1 Only the worst-case scenario is considered. No positive effects of the search operators are considered. This has lead to the development of Exact Schema Theorems. Problem 2 The theorem concentrates on the number of schema surviving not which schema survive. Such considerations have been addressed by the utilization of Markov chains to provide models of behavior associated with specific individuals in the population. 36 / 37
  89. 89. Problems Problem 1 Only the worst-case scenario is considered. No positive effects of the search operators are considered. This has lead to the development of Exact Schema Theorems. Problem 2 The theorem concentrates on the number of schema surviving not which schema survive. Such considerations have been addressed by the utilization of Markov chains to provide models of behavior associated with specific individuals in the population. 36 / 37
  90. 90. Problems Problem 3 Claims of “exponential increases” in fit schema i.e., if the expectation operator of Schema Theorem is ignored and the effects of crossover and mutation discounted, the following result was popularized by Goldberg, m(H, t + 1)≥(1 + c)m(H, t) where c is the constant by which fit schema are always fitter than the population average. PROBLEM!!! Unfortunately, this is rather misleading as the average population fitness will tend to increase with t, thus population and fitness of remaining schema will tend to converge with increasing ‘time’. 37 / 37
  91. 91. Problems Problem 3 Claims of “exponential increases” in fit schema i.e., if the expectation operator of Schema Theorem is ignored and the effects of crossover and mutation discounted, the following result was popularized by Goldberg, m(H, t + 1)≥(1 + c)m(H, t) where c is the constant by which fit schema are always fitter than the population average. PROBLEM!!! Unfortunately, this is rather misleading as the average population fitness will tend to increase with t, thus population and fitness of remaining schema will tend to converge with increasing ‘time’. 37 / 37

×