• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Not Your Grandmother's Genetic Algorithm

Not Your Grandmother's Genetic Algorithm



Talk presented in honor of John Holland at Adapation, Order & Emergence http://www.ntu.edu.sg/event/Pages/AdaptationOrderAndEmergence.aspx.

Talk presented in honor of John Holland at Adapation, Order & Emergence http://www.ntu.edu.sg/event/Pages/AdaptationOrderAndEmergence.aspx.



Total Views
Views on SlideShare
Embed Views



4 Embeds 20

http://blog.coevolver.net 13
http://www.coevolver.net 3
http://www.slideshare.net 3
http://www.linkedin.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Honor Escaping Hierarchical Traps with Competent GAs

Not Your Grandmother's Genetic Algorithm Not Your Grandmother's Genetic Algorithm Presentation Transcript

  • Not Your Grandmother’s Genetic Algorithm David E. Goldberg Illinois Genetic Algorithms Laboratory University of Illinois at Urbana-Champaign Urbana, IL 61801 USA Email: [email_address] ; Web: http://www.illigal.uiuc.edu
  • GAs Had Their Warhol 15, Right?
    • Evolution timeless, GAs so 90s.
    • First-generation GA results were mixed in practice.
    • Sometimes worked, sometimes not & first impressions stuck.
    • But John Holland’s ideas had legs.
    • In 90s, logical continuation of John’s thinking has led to
      • Completion of theory in certain sense,
      • & GAs that solve large, hard problems quickly, reliably, and accurately.
    • Consider ways today’s procedures are not your grandmother’s GA.
    • Along way, reflect on lessons learned from John.
    Andy Warhol (1928-1987)
  • Roadmap
    • A technoscientific fairy tale.
    • 3 things I learned from John.
    • The one-minute genetic algorithmist.
    • The unreasonableness of GAs.
    • Not your grandmother’s GA:
      • Holland theory in design.
      • A race & GA convergence.
      • A billion bits or bust.
  • A Technoscientific Fairy Tale
    • Once upon a time…
      • There was a civil engineer
      • working for Stoner Associates
      • doing hydraulics software for pipelines.
    • Was starting to do real-time control &
      • wondered how human operators
      • controlled gas pipelines
      • like you or I drive a car.
    • Was getting restless and just read a book.
    • Called Ben Wylie in Civil Engineering at Michigan and got 25% RA and went back to school.
  • One Fine Day in A 2 in Fall 1980
    • First day of classes and was signed up for standard AI course.
    • Expert systems were the rage, Prolog was hip, LISP was cool.
    • Class was cancelled with little sign on the door.
    • Hopes and dreams down the drain.
    • Searched and searched for a replacement.
    • Found CCS 524, Intro to Adaptive Systems , taught by John Holland.
  • Some Prof Named Holland
    • Youngish looking prof:
      • Talking about biology & genetics.
      • Samuel’s checker player.
      • Schemas and building blocks.
      • Classifier systems.
    • What’s nice civil engineer doing in class like this?
    • When was Prof Holland going to get to real AI I could use for pipelines?
    • Or maybe this was the real AI.
  • 3 Things I Learned From John
    • Learned many things from John.
    • Later will reflect on some specifics.
    • Now, 3 meta-lessons:
      • Start good science with good story.
      • Go broad or go home.
      • Adopt a Will Rogers theory of modeling.
  • Good Stories  Good Science
    • Johns method: Conceptual story  some math  computation.
    • John the best conceptual story teller I know.
    • Spins self-contained Quine-like “webs of knowledge.”
    • John’s webs invent things we don’t yet know, but soon will.
    • Stories have persuasive coherence that sustains effort when going gets tough.
  • Go Broad or Go Home
    • Many talk a good interdisciplinary game.
    • Don’t know interdisciplinary until you hang out with John.
    • Fearless in plucking results from X (X = math, CS, econ, psych, biology, linguistics, poetry, philosophy, etc.)
    • John’s students spoiled.
    • Hard to work in typical university after working with John.
  • Will Rogers Theory of Modeling
    • Will Rogers theory of models: “I never met a model I didn’t like.”
    • John likes all kinds of models.
    • CS notion of theory has become sterile.
    • Theory as sometimes instrumental to thought, not always end goal.
    Will Rogers (1879-1935)
  • A Model of Models Error, ε Cost of Modeling, C Engineer/Inventor Scientist/Mathematician
  • Marginal Analysis
    • When should engineer/inventor adopt more expensive model?
    • At the margins, when Δ B ≥ Δ C.
    • Marginal benefit of model to technology under development must equal or exceed its marginal cost.
    • To engineer/inventor, artifact is the object of study  models almost always instrumental.
    • To scientist/mathematician building a model
      • may be the object
      • or instrumental to some other goal (then engineer’s calculus applies).
  • What is a “Model?” The Modeling Spectrum Low Cost/ High Error High Cost/ Low Error Unarticulated Wisdom Articulated Qualitative Model Dimensional Models Facetwise Models Equations of Motion
  • One-Minute Genetic Algorithmist
    • What is a GA?
    • Solutions as chromosomes.
    • Means of evaluating fitness to purpose.
    • Create initial population.
    • Apply selection and genetic operators:
      • Survival of the fittest.
      • Mutation
      • Crossover
    • Repeat until good enough.
    • Puzzle: operators by themselves uninteresting.
  • Crossover Alone Uninteresting
    • Combine bits and pieces of good parents.
    • Speculate on, new, possibly better children.
    • By itself, random shuffle.
    • Gedanken experiments for other ops.
    • Example, one-point X:
    Before X After X 11111 11000 00000 00111
  • The Unreasonableness of GAs
    • How do individually uninteresting operators yield interesting behavior?
    • Others will talk about emergence.
    • 1983 innovation intuition: Genetic algorithm power like that of human innovation.
    • Separate
      • Selection + mutation as hillclimbing or kaizen.
      • Selection + recombination  Let’s examine.
    • Different modes or facets of innovation or invention.
  • Selection + Recombination = Innovation
    • Combine notions to form ideas.
    • It takes two to invent anything. The one makes up combinations; the other chooses, recognizes what he wishes and what is important to him in the mass of the things which the former has imparted to him . P. Valéry
    Paul Valéry (1871-1945)
  • Holland Theory in Design
    • Many GAs don’t scale & much GA theory inapplicable.
    • Need design theory that works:
      • Understand building blocks (BBs), notions or subideas.
      • Ensure BB supply.
      • Ensure BB growth.
      • Control BB speed.
      • Ensure good BB decisions.
      • Ensure good BB mixing (exchange).
      • Know BB challengers.
    • Can use theory to design scalable & efficient GAs.
  • A Sense of Time
    • Truncation selection: make s copies each of top 1/ s th of the population.
    • P(t+1) = s P(t) until P(t) = 1
    • P(t) = s t P(0)
    • Solve for takeover time t *: time to go from one good guy to all good guys (or all but one).
    • t * = ln n / ln s
  • So What?
    • Who cares about selection alone?
    • I want to analyze a “real GA”.
    • How can selection-only analysis help me?
    • Answer: Imagine another characteristic time, the innovation or mixing time.
  • The Innovation Time, t i
    • Innovation time is the average time to create an individual better than one so far.
    • Under crossover imagine pi, the probability of recomb event creating better individual.
    • Innovation probability in Goldberg, Deb & Thierens (1993) and Thierens & Goldberg (1993).
  • Schematic of the Race
  • Golf Clubs Have Sweet Spots
    • So do GAs.
    • Easy problems, big sweet spots.
    • Monkey can set GA parameters.
    • Hard problems, vanishing sweet spots.
    [Goldberg, Deb, & Theirens, 1993]
  • My Dr. Evil Moment
    • Lunchtime question: do real large problems draw attention to theoretical & design findings?
    • Dr. Evil’s mistake: Wondered if GAs could go to 10 6 vars.
    • Decided to go for a billion.
    • Use simple underlying problem (OneMax) with Gaussian noise (0.1 variance of deterministic problem)
    • Don’t try this at home!!!
    We get the warhead and then hold the world ransom for... 1 MILLION DOLLARS !
  • Road to Billion Paved with Speedups
    • Naïve implementation: 100 terabytes & 2 72 random number calls.
    • cGA  Memory O(ℓ) v. O(ℓ 1.5 ).
    • Parallelization  speedup n p .
    • Vectorize four bits at a time  speedup 4.
    • Other doodads (bitwise ops, limit flops, inline fcns, precomputed evals)  speedup 15.
    • Gens & pop size scale as expected.
  • A Billion Bits or Bust
    • Simple hillclimber solves 1.6(10 4 ) or (2 14 ).
    • Souped-up cGA solves 33 million (2 25 ) to full convergence.
    • Solves 1.1 billion (2 30 ) with relaxed convergence.
    • Growth rate the same  Solvable to convergence.
  • Not Your Grandmother’s GA
    • GA design advanced by taking John’s ideas and running with them.
    • Large difficult problems in grasp.
    • Theory and practice in sync.
    • These direct lessons are crucial.
    • Meta-lessons of Holland’s thinking as important for future for complex systems & interdisciplinary work, generally.
  • More Information
    • Goldberg, D. E. (2002). The design of innovation: Lessons from and for competent genetic algorithms. Boston, MA: Kluwer Academic Publishers.
    • Lab: http://www-illigal.ge.uiuc.edu/
    • DISCUS: http://www-discus.ge.uiuc.edu /
    • iFoundry: http://ifoundry.illigal.uiuc.edu /
    • WPE-2008: http://www-illigal.ge.uiuc.edu/wpe