Your SlideShare is downloading. ×
Reflections of a Keen Modeler
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Reflections of a Keen Modeler


Published on

OBUPM-2007 presentation by David E. Goldberg

OBUPM-2007 presentation by David E. Goldberg

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Some Reflections of a Keen Modeler David E. Goldberg Illinois Genetic Algorithms Laboratory University of Illinois at Urbana-Champaign Urbana, Illinois 61801 [email_address]
  • 2. A Life in Models
    • What should I talk about?
    • Started engineering college in 1971.
    • Calculus was cool, and differential equations were amazing
    • Really dug making models with computers.
    • Bumped into fluid mechanics my junior year at Michigan.
    • Didn’t understand impact of that collision then.
    • Started to understand it when I bumped into GAs in 1980.
    • Bumped into more models as chief scientist of web startup.
    • Reflections on life in models.
  • 3. Roadmap
    • What’s a model?
    • An economic model of models.
    • Models in GAs.
    • From GAs to PMBGAs/EDAs.
    • Models in PMBGA/EDAs.
    • Models in the education of the creative genetic algorithmist.
  • 4. What is a Model?
    • A model is a system that represents one or more facets of some other system.
    • Typical model  facet combinations:
      • Drawing or solid model  geometry.
      • Prototype  geometry & operation.
      • Graph  variation of variable with independent variable (time, space, etc.).
      • Equilibrium equation  select state variables at steady state.
      • Dynamic equation  variation of select state variables with time.
      • Simulation  similar to equations, but uses an intermediate artifact to calculate..
  • 5. Words and Models
    • Foregoing examples typical engineering models.
    • Language is also used in modeling.
    • Many first models are verbal.
    • Types of verbal models:
      • Single word or noun phrase.
      • Description of an object/process.
      • Feature list.
      • Dimension list.
      • Set of engineering specifications.
  • 6. An Economy of Models
    • Have many models with different precision-accuracy and different costs.
    • Can we evaluate model usage rationally?
    • Consider economics of intellection and spectrum of models.
  • 7. Fundamental Modeling Tradeoff
    • Error versus cost of modeling
    ε , Error C, Cost of Modeling Engineer/Inventor Scientist/Mathematician
  • 8. Marginal Analysis
    • Optimal thinking, when marginal cost of a thought equals marginal benefit to design.
    •  C =  B
    • If cost higher than advance in design, thinking is uneconomic.
    • Science: model is the end.
    • Engineering; model is instrumental.
    • Match model complexity to ends required.
  • 9. A Spectrum of Models The Modeling Spectrum Low Cost/ High Error High Cost/ Low Error Unarticulated Wisdom Articulated Qualitative Model Dimensional Models Facetwise Models Equations of Motion
  • 10. Disciplines: Mature & Immature
    • Mature disciplines are mature partially because they have developed sense of models.
    • GAs circa 1980: Immature discipline.
    • Few models available.
    • Hodgepodge of religious beliefs (xover v. mutation, etc.).
    • Some qualitative notions of what is important.
  • 11. What Are Models Good For?
    • Many uses for models:
      • Description: describe the ways things are (were).
      • Prediction: describe the ways things will be.
      • Prescription: describe the way things should be.
    • Key variables: time and change.
    • Usually assumes have extant object to model.
    • Oftentimes assumes extant models.
  • 12. Crossing Qual-Quant Divide
    • Needed quantitative understanding of qualitative theories/experience.
    • How do you cross the qual-quant divide?
    • Do not expect full equations of motion to provide answers.
    • Need little models of important tradeoffs.
  • 13. Models in GAs
    • Our interest is in models that are
      • Principled
      • Quantitative
      • Not equations of motion
    • Call them little models , where “little” is a term of approbation.
    • Little models simplified, not universal.
    • Where do little models come from?
  • 14. Sources of Little Models
    • Dimensional analysis (more in moment)
    • Construction of model for single facet.
    • Reduction of equation of motion for one or small number of facets.
    • Incorporation of qualitative reasoning.
    • Extremum principle.
    • Data usage (data-influenced), but empirical fit alone not enough.
  • 15. Aside on Dimensional Analysis
    • Common in physics and engineering, but not CS.
    • Buckingham PI theorem: n dimensional parameters, m dimensions  n – m independent dimensionless parameters.
    • [ V ] = L/T, [ D ] = L, [ ν ] = L 2 /T n = 3, m = 2, n – m = 1 R = VD / ν , the Reynolds number
    • One difference: many CS parameters are pure numbers. Must derive time, spatial, or other scale, then renormalize.
    • Can play the same game for GAs, AI, & AL.
  • 16. A LM of Selection Alone
    • Choose truncation selection because it is easy to analyze.
    • Truncation selection: make s copies each of top 1/ s th of the population.
    • Want to know time for good individual to “takeover” population: takeover time.
  • 17. Takeover Time Under Truncation
    • Let P be proportion of best individuals.
    • How many more next generation? s
    • P ( t +1) = s P ( t ) until P ( t )=1.
    • P ( t ) = s t P (0).
    • Solve for takeover time t *: time to go from one good guy to all or all but one good guys.
    • t* = ln n/ ln s
  • 18. So What?
    • Who cares about selection alone?
    • I want to analyze a “real GA.”
    • How can selection only analysis help me?
    • Answer: Imagine another characteristic time, the innovation time.
  • 19. A LM of Innovation Time
    • Assume using crossover-based innovation with probability p c .
    • Assume that crossover event gives improvement with probability p i in population of size n.
    • Can calculate the expected time of arrival of next improvement.
  • 20. The Innovation Time, t i
    • Define innovation time as the expected time to create an individual better than one so far.
    • t i = ( p c p i n ) -1
    • Model is facetwise, probabilistic & incomplete ( p i unknown).
    • p i estimated in Goldberg, Deb, & Thierens (1993) and Thierens & Goldberg (1993).
  • 21. The Race: Integrating 2 Models
    • Have two facetwise models, but want integrated understanding.
    • Putting models in t terms gives us an idea.
    • Consider relative magnitudes of the two times: a dimensional argument.
    • Consider which is favorable to innovation: a qualitative argument.
  • 22. Schematic of the Race
  • 23. Dimensional & Qualitative Argument
    • This argument results in an integrated little model.
    • Define innovation number G = t* / t i
    • G = p c p i n ln n/ ln s.
    • Want takeover time greater than innovation time or G > 1.
    • Quantity like Reynolds number in fluids.
  • 24. A Simple Control Map
    • Draw success region in GA parameter space.
    • Control map.
    • p c > c ( m, n ) ln s.
    • For p c versus ln s a straight line .
  • 25. 1993 Empirical Result
    • Easy problems are no problem.
    • GA has a large “sweet spot”.
    • A monkey can set cross probability & selection pressure
    [Goldberg, Deb, & Theirens, 1993]
  • 26. Lessons of LMing
    • Facetwise approach is fast.
    • Requires some skill in identifying correct facets and in integrating them.
    • Payoff in understanding is out of proportion to complexity of the effort.
    • Do not waste time on needless parametric study.
    • Verify expected behavior and move on.
    • Facetwise understanding improves pedagogy and ability to explain things simply.
  • 27. Models in PMBGA/EDAs
    • Baluja & others recognized key thing that Holland had identified earlier.
    • Population + genetic operators  Probability distribution over possible structures.
    • Early PMBGA/EDAs used fixed models.
    • Later ones built models.
    • What is going on?
  • 28. Primary Effect: Good Solutions Current population Selection New population Bayesian network
  • 29. Secondary: Structural Knowledge
    • Mental models give us flexibility to think about world without paying big price.
    • Built models can be used to speed up evolution of good structures:
      • Parallelism
      • Time continuation
      • Hybridization
      • Evaluation relaxation
    • Usually think of them independently.
  • 30. Supermultiplicative Speedup
    • Speed-Up: Ratio of # function evaluations without efficiency enhancement to that with it.
    • Only 1-15% individuals need evaluation
    • Speed-Up: 30–53
    • Have decision tree & ECGA versions.
    Fitness modeling in BOA
  • 31. Use of Structural/Fitness Surrogates
    • Parallelism: decomposition of problem for efficient parallelism.
    • Time continuation: understanding of problem sequencing and parameterization.
    • Hybridization: What neighborhood operators appropriate when & information for optimal division of labor.
    • Estimation of algorithm parameters.
  • 32. Important Frontier PMBGA/EDAs
    • Extending the notion of model building.
    • Pervade every aspect of algorithm coordination.
    • Already movement in that direction.
    • More still needs to be done.
  • 33. Education of Creative GAmist
    • Early 60s-80s pioneering years.
    • Years where categories genetic algorithms and evolutionary computation were created.
    • How do we rekindle that creativity moving forward?
    • How do we educate creative genetic algorithmists of the future?
    • Crossed qual-quant divide.
    • Need to go back to qual.
  • 34. Key Distinction
    • Modeling of imagined or desired objects versus extant objects.
    • What can we draw on?
      • Existing objects that fail in some regard.
      • Similar or related objects.
      • Analogically related objects.
      • Creatively concocted objects.
    • Takes us back to the tabula rasa problem.
    • How do we even discuss that which does not exist?
  • 35. Tabula Rasa: Curse & Blessing of Category Creator
    • How do we design when we don’t know how to talk about what we are designing?
    • Let’s start at the human beginnings of conceptual clarity.
    • Let’s start at the beginning of formal philosophy.
    • Let’s start with two key techniques from Athens.
  • 36. What Examples of New Thought?
    • Clearest examples are from philosophy.
    • Presocratic  Socrates  Plato  Aristotle.
    • Mechanisms of the new thought:
      • Socratic dialectic
      • Aristotelian data mining
  • 37. Socrates and Dialectic
    • Socrates was a pain in the neck.
    • Walked around Athens asking everyone impossible questions.
    • Then proved their answers were wrong, but rarely gave an answer himself.
    • Nonetheless, Socrates’s method was useful.
    • Conversation trying to probe what things really are (or might be).
    • Questions were the rights ones. Whitehead’s famous remark.
    Socrates (470-399 BCE)
  • 38. The Probing of Dialectic
    • Questions directed at the essence of things.
    • What is the meaning of a common phrase? “What is virtue?”
    • Answers often betray our lack of knowledge and understanding.
    • Examine answers critically, often with more questions.
    • Ask penetrating questions about the answers.
  • 39. What’s This Got to Do with GAs?
    • Questions & conversation is at roots of new inventions.
    • Research on tech visionaries shows that problem finding is the main activity of successful TVs
    • Spark of insight may come as flash, but dialectic necessary in new product creation.
    • Three roles of questions:
      • Probe field needs.
      • Probe biases or politics of the field..
      • Probe developmental hurdles.
  • 40. Tactics of Dialectic
    • Critical:
      • Equivocation: use of term in different senses.
      • Question begging: assuming the conclusion.
      • Infinite regress: infinite sequence implying incoherence.
      • Loss of contrast & emptiness: distinction with little or no difference.
    • Creative:
      • Definition: Seek essential distinctions.
      • Analogies: Certain dialectical similarities.
      • Thought experiments: Hypothetical w/ true premises that does not follow.
  • 41. Aristotelian Data Mining
    • Called The Philosopher by some.
    • Amazing range and scope of work.
    • Created many of basic categories of college curriculum.
    • Founded a school the Lyceum.
    • We have 1/3 his output (2000 pages in 30 books).
    • Categories and Metaphysics.
    • Method very modern:
      • Empirical search for data.
      • Considered attributes, which he named.
      • Classified data according to his attributes.
    Aristotle (384-322 BCE)
  • 42. Analysis is More than Pretty Math
    • Sons & daughters of Newton spoiled by equations of motion.
    • Put too much faith in sets of equations.
    • Complexity demands heavy lifting of the facets for understanding.
    • Range of modeling skills, particularly qualitative reasoning.
    • Necessary for creative activity that is genetic and evolutionary computation.
  • 43. Summary
    • Models generally:
      • Economy of models.
      • Modeling spectrum.
    • Models in GAs w/ emphasis on little models.
    • Models in PMBGA/EDAs.
    • Models in education of the creative genetic algorithmist.
  • 44. Conclusions
    • Modeling is central to our business.
    • But simple view of modeling limits progress.
    • Need sophisticated, flexible approach to advance state of the art most quickly.
    • Still many opportunities for model advance in PMBGA/EDAs.
    • Mastery of the spectrum of models from qual to quant, from little to equations of motion holds hope for continued vibrancy and creativity of the field.
  • 45. More Information
    • DoI, the book
    • TEE, the book.
    • TEE, the blog.
    • TEE, the course.
    • MTV, the course.
    • GAs, the course
    • 2007 Workshop on Philosophy & Engineering (WPE) http://www-
    • Illinois Genetic Algorithms Lab