Upcoming SlideShare
×

# Reflections of a Keen Modeler

• 1,807 views

OBUPM-2007 presentation by David E. Goldberg

OBUPM-2007 presentation by David E. Goldberg

• Comment goes here.
Are you sure you want to
Be the first to comment

Total Views
1,807
On Slideshare
0
From Embeds
0
Number of Embeds
0

Shares
0
0
Likes
3

No embeds

### Report content

No notes for slide

### Transcript

• 1. Some Reflections of a Keen Modeler David E. Goldberg Illinois Genetic Algorithms Laboratory University of Illinois at Urbana-Champaign Urbana, Illinois 61801 [email_address]
• 2. A Life in Models
• What should I talk about?
• Started engineering college in 1971.
• Calculus was cool, and differential equations were amazing
• Really dug making models with computers.
• Bumped into fluid mechanics my junior year at Michigan.
• Didn’t understand impact of that collision then.
• Started to understand it when I bumped into GAs in 1980.
• Bumped into more models as chief scientist of web startup.
• Reflections on life in models.
• What’s a model?
• An economic model of models.
• Models in GAs.
• From GAs to PMBGAs/EDAs.
• Models in PMBGA/EDAs.
• Models in the education of the creative genetic algorithmist.
• 4. What is a Model?
• A model is a system that represents one or more facets of some other system.
• Typical model  facet combinations:
• Drawing or solid model  geometry.
• Prototype  geometry & operation.
• Graph  variation of variable with independent variable (time, space, etc.).
• Equilibrium equation  select state variables at steady state.
• Dynamic equation  variation of select state variables with time.
• Simulation  similar to equations, but uses an intermediate artifact to calculate..
• 5. Words and Models
• Foregoing examples typical engineering models.
• Language is also used in modeling.
• Many first models are verbal.
• Types of verbal models:
• Single word or noun phrase.
• Description of an object/process.
• Feature list.
• Dimension list.
• Set of engineering specifications.
• 6. An Economy of Models
• Have many models with different precision-accuracy and different costs.
• Can we evaluate model usage rationally?
• Consider economics of intellection and spectrum of models.
• Error versus cost of modeling
ε , Error C, Cost of Modeling Engineer/Inventor Scientist/Mathematician
• 8. Marginal Analysis
• Optimal thinking, when marginal cost of a thought equals marginal benefit to design.
•  C =  B
• If cost higher than advance in design, thinking is uneconomic.
• Science: model is the end.
• Engineering; model is instrumental.
• Match model complexity to ends required.
• 9. A Spectrum of Models The Modeling Spectrum Low Cost/ High Error High Cost/ Low Error Unarticulated Wisdom Articulated Qualitative Model Dimensional Models Facetwise Models Equations of Motion
• 10. Disciplines: Mature & Immature
• Mature disciplines are mature partially because they have developed sense of models.
• GAs circa 1980: Immature discipline.
• Few models available.
• Hodgepodge of religious beliefs (xover v. mutation, etc.).
• Some qualitative notions of what is important.
• 11. What Are Models Good For?
• Many uses for models:
• Description: describe the ways things are (were).
• Prediction: describe the ways things will be.
• Prescription: describe the way things should be.
• Key variables: time and change.
• Usually assumes have extant object to model.
• Oftentimes assumes extant models.
• 12. Crossing Qual-Quant Divide
• Needed quantitative understanding of qualitative theories/experience.
• How do you cross the qual-quant divide?
• Do not expect full equations of motion to provide answers.
• Need little models of important tradeoffs.
• 13. Models in GAs
• Our interest is in models that are
• Principled
• Quantitative
• Not equations of motion
• Call them little models , where “little” is a term of approbation.
• Little models simplified, not universal.
• Where do little models come from?
• 14. Sources of Little Models
• Dimensional analysis (more in moment)
• Construction of model for single facet.
• Reduction of equation of motion for one or small number of facets.
• Incorporation of qualitative reasoning.
• Extremum principle.
• Data usage (data-influenced), but empirical fit alone not enough.
• 15. Aside on Dimensional Analysis
• Common in physics and engineering, but not CS.
• Buckingham PI theorem: n dimensional parameters, m dimensions  n – m independent dimensionless parameters.
• [ V ] = L/T, [ D ] = L, [ ν ] = L 2 /T n = 3, m = 2, n – m = 1 R = VD / ν , the Reynolds number
• One difference: many CS parameters are pure numbers. Must derive time, spatial, or other scale, then renormalize.
• Can play the same game for GAs, AI, & AL.
• 16. A LM of Selection Alone
• Choose truncation selection because it is easy to analyze.
• Truncation selection: make s copies each of top 1/ s th of the population.
• Want to know time for good individual to “takeover” population: takeover time.
• 17. Takeover Time Under Truncation
• Let P be proportion of best individuals.
• How many more next generation? s
• P ( t +1) = s P ( t ) until P ( t )=1.
• P ( t ) = s t P (0).
• Solve for takeover time t *: time to go from one good guy to all or all but one good guys.
• t* = ln n/ ln s
• 18. So What?
• Who cares about selection alone?
• I want to analyze a “real GA.”
• How can selection only analysis help me?
• Answer: Imagine another characteristic time, the innovation time.
• 19. A LM of Innovation Time
• Assume using crossover-based innovation with probability p c .
• Assume that crossover event gives improvement with probability p i in population of size n.
• Can calculate the expected time of arrival of next improvement.
• 20. The Innovation Time, t i
• Define innovation time as the expected time to create an individual better than one so far.
• t i = ( p c p i n ) -1
• Model is facetwise, probabilistic & incomplete ( p i unknown).
• p i estimated in Goldberg, Deb, & Thierens (1993) and Thierens & Goldberg (1993).
• 21. The Race: Integrating 2 Models
• Have two facetwise models, but want integrated understanding.
• Putting models in t terms gives us an idea.
• Consider relative magnitudes of the two times: a dimensional argument.
• Consider which is favorable to innovation: a qualitative argument.
• 22. Schematic of the Race
• 23. Dimensional & Qualitative Argument
• This argument results in an integrated little model.
• Define innovation number G = t* / t i
• G = p c p i n ln n/ ln s.
• Want takeover time greater than innovation time or G > 1.
• Quantity like Reynolds number in fluids.
• 24. A Simple Control Map
• Draw success region in GA parameter space.
• Control map.
• p c > c ( m, n ) ln s.
• For p c versus ln s a straight line .
• 25. 1993 Empirical Result
• Easy problems are no problem.
• GA has a large “sweet spot”.
• A monkey can set cross probability & selection pressure
[Goldberg, Deb, & Theirens, 1993]
• 26. Lessons of LMing
• Facetwise approach is fast.
• Requires some skill in identifying correct facets and in integrating them.
• Payoff in understanding is out of proportion to complexity of the effort.
• Do not waste time on needless parametric study.
• Verify expected behavior and move on.
• Facetwise understanding improves pedagogy and ability to explain things simply.
• 27. Models in PMBGA/EDAs
• Baluja & others recognized key thing that Holland had identified earlier.
• Population + genetic operators  Probability distribution over possible structures.
• Early PMBGA/EDAs used fixed models.
• Later ones built models.
• What is going on?
• 28. Primary Effect: Good Solutions Current population Selection New population Bayesian network
• 29. Secondary: Structural Knowledge
• Mental models give us flexibility to think about world without paying big price.
• Built models can be used to speed up evolution of good structures:
• Parallelism
• Time continuation
• Hybridization
• Evaluation relaxation
• Usually think of them independently.
• 30. Supermultiplicative Speedup
• Speed-Up: Ratio of # function evaluations without efficiency enhancement to that with it.
• Only 1-15% individuals need evaluation
• Speed-Up: 30–53
• Have decision tree & ECGA versions.
Fitness modeling in BOA
• 31. Use of Structural/Fitness Surrogates
• Parallelism: decomposition of problem for efficient parallelism.
• Time continuation: understanding of problem sequencing and parameterization.
• Hybridization: What neighborhood operators appropriate when & information for optimal division of labor.
• Estimation of algorithm parameters.
• 32. Important Frontier PMBGA/EDAs
• Extending the notion of model building.
• Pervade every aspect of algorithm coordination.
• Already movement in that direction.
• More still needs to be done.
• 33. Education of Creative GAmist
• Early 60s-80s pioneering years.
• Years where categories genetic algorithms and evolutionary computation were created.
• How do we rekindle that creativity moving forward?
• How do we educate creative genetic algorithmists of the future?
• Crossed qual-quant divide.
• Need to go back to qual.
• 34. Key Distinction
• Modeling of imagined or desired objects versus extant objects.
• What can we draw on?
• Existing objects that fail in some regard.
• Similar or related objects.
• Analogically related objects.
• Creatively concocted objects.
• Takes us back to the tabula rasa problem.
• How do we even discuss that which does not exist?
• 35. Tabula Rasa: Curse & Blessing of Category Creator
• How do we design when we don’t know how to talk about what we are designing?
• Let’s start at the human beginnings of conceptual clarity.
• Let’s start at the beginning of formal philosophy.
• 36. What Examples of New Thought?
• Clearest examples are from philosophy.
• Presocratic  Socrates  Plato  Aristotle.
• Mechanisms of the new thought:
• Socratic dialectic
• Aristotelian data mining
• 37. Socrates and Dialectic
• Socrates was a pain in the neck.
• Walked around Athens asking everyone impossible questions.
• Then proved their answers were wrong, but rarely gave an answer himself.
• Nonetheless, Socrates’s method was useful.
• Conversation trying to probe what things really are (or might be).
• Questions were the rights ones. Whitehead’s famous remark.
Socrates (470-399 BCE)
• 38. The Probing of Dialectic
• Questions directed at the essence of things.
• What is the meaning of a common phrase? “What is virtue?”
• Answers often betray our lack of knowledge and understanding.
• Examine answers critically, often with more questions.
• 39. What’s This Got to Do with GAs?
• Questions & conversation is at roots of new inventions.
• Research on tech visionaries shows that problem finding is the main activity of successful TVs
• Spark of insight may come as flash, but dialectic necessary in new product creation.
• Three roles of questions:
• Probe field needs.
• Probe biases or politics of the field..
• Probe developmental hurdles.
• 40. Tactics of Dialectic
• Critical:
• Equivocation: use of term in different senses.
• Question begging: assuming the conclusion.
• Infinite regress: infinite sequence implying incoherence.
• Loss of contrast & emptiness: distinction with little or no difference.
• Creative:
• Definition: Seek essential distinctions.
• Analogies: Certain dialectical similarities.
• Thought experiments: Hypothetical w/ true premises that does not follow.
http://www.amazon.com/Practice-Philosophy-Handbook-Beginners-3rd/dp/0132308487
• 41. Aristotelian Data Mining
• Called The Philosopher by some.
• Amazing range and scope of work.
• Created many of basic categories of college curriculum.
• Founded a school the Lyceum.
• We have 1/3 his output (2000 pages in 30 books).
• Categories and Metaphysics.
• Method very modern:
• Empirical search for data.
• Considered attributes, which he named.
• Classified data according to his attributes.
Aristotle (384-322 BCE)
• 42. Analysis is More than Pretty Math
• Sons & daughters of Newton spoiled by equations of motion.
• Put too much faith in sets of equations.
• Complexity demands heavy lifting of the facets for understanding.
• Range of modeling skills, particularly qualitative reasoning.
• Necessary for creative activity that is genetic and evolutionary computation.
• 43. Summary
• Models generally:
• Economy of models.
• Modeling spectrum.
• Models in GAs w/ emphasis on little models.
• Models in PMBGA/EDAs.
• Models in education of the creative genetic algorithmist.
• 44. Conclusions
• Modeling is central to our business.
• But simple view of modeling limits progress.
• Need sophisticated, flexible approach to advance state of the art most quickly.
• Still many opportunities for model advance in PMBGA/EDAs.
• Mastery of the spectrum of models from qual to quant, from little to equations of motion holds hope for continued vibrancy and creativity of the field.