THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS David E. Goldberg Department of General Engine...
Innovation This & Innovation That <ul><li>The business world is abuzz with “innovation.” </li></ul><ul><li>Popular books t...
Genetic Algorithms are Coming of Age <ul><li>Genetic algorithms usage growing since 1985.  </li></ul><ul><li>Used to desig...
Connect the Dots <ul><li>Wouldn’t it be nice to understand innovation scientifically? </li></ul><ul><li>GAs that solve pro...
Overview <ul><li>One Minute Genetic Algorithmist. </li></ul><ul><li>GAs and Innovation. </li></ul><ul><li>A lesson from th...
Who Am I? <ul><li>General Engineering, University of Illinois at Urbana-Champaign, 1990-2002. </li></ul><ul><li>Engineerin...
Course Text <ul><li>Goldberg, D. E. (2002).  The design of innovation: Lessons from and for competent genetic algorithms. ...
Background GA Stuff <ul><li>Course is self-contained.  Requires maturity of engineering, science, or math/CS BS degree. </...
Today’s Agenda <ul><li>One-minute genetic algorithmist. </li></ul><ul><li>The innovation intuition. </li></ul><ul><li>Intr...
The One-Minute Genetic Algorithmist <ul><li>What is a GA? </li></ul><ul><li>Solutions as  chromosomes. </li></ul><ul><li>M...
What is a Genetic Algorithm? <ul><li>A genetic algorithm is a search procedure based on the mechanics of natural selection...
Solutions as Chromosomes <ul><li>Code solutions as artificial chromosomes. </li></ul><ul><li>Possible strings = possible s...
Determining Fitness <ul><li>Something must decide good from bad. </li></ul><ul><li>Possibilities: </li></ul><ul><ul><li>Ob...
Operators <ul><li>Selection  </li></ul><ul><li>Recombination </li></ul><ul><li>Mutation </li></ul>
Selection <ul><li>Darwinian survival of the fittest. </li></ul><ul><li>Give more copies to better guys. </li></ul><ul><li>...
Recombination <ul><li>Combine bits and pieces of good parents. </li></ul><ul><li>Speculate on, new, possibly better childr...
Mutation <ul><li>Mutation is a random alteration of a string. </li></ul><ul><li>Change a gene, small movement in the neigh...
GAs & Innovation:  The Fundamental Intuition <ul><li>How do individually  uninteresting  operators yield  interesting  beh...
Selection+Mutation=Improvement <ul><li>Total Quality Management: continual improvement; Japanese term:  kaizen. </li></ul>...
Selection+Recombination=Innovation <ul><li>Combine  notions  to form  ideas. </li></ul><ul><li>“ Indeed, it is obvious tha...
Two Facets of Innovation <ul><li>Selection+mutation = hillclimbing or improvement. </li></ul><ul><li>Selection+crossover =...
4 Intriguing Applications <ul><li>GE designs a Boeing 777 jet engine.  </li></ul><ul><li>John Deere schedules a factory. <...
GE Designs a Jet Engine <ul><li>GA work started in late 80s. </li></ul><ul><li>Found 2% efficiency increase w/ GA + expert...
A GA Schedules for John Deere <ul><li>Optimax created scheduling program. </li></ul><ul><li>Schedules planter line and 5 o...
Portfolio Management with GA <ul><li>First Quadrant manages $28B with daily help from a GA. </li></ul><ul><li>Chromosomes ...
NMSU, the Beauty & the Beast <ul><li>GA replaces criminal sketch artist. </li></ul><ul><li>NMSU system called  Faceprints....
GAs Have Special Kind of Appeal <ul><li>Going boldly where optimization has and has not gone. </li></ul><ul><li>From engin...
Summary <ul><li>One-minute genetic algorithmist. </li></ul><ul><li>The paradox of uninteresting operators. </li></ul><ul><...
Conclusions <ul><li>Innovation is important, but mysterious. </li></ul><ul><li>GAs are intriguing, but first-generation GA...
THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS Produced by The Office of Continuing  Engineer...
Upcoming SlideShare
Loading in …5
×

Design of Innovation: Innovation & Genetic Algorithms

6,740 views

Published on

This is the introductory module of a short course at the UIUC called The Design of Innovation: Lessons from and for Genetic Algorithms. It covers the material of the book of the same name.

Published in: Business, Education
0 Comments
20 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,740
On SlideShare
0
From Embeds
0
Number of Embeds
71
Actions
Shares
0
Downloads
0
Comments
0
Likes
20
Embeds 0
No embeds

No notes for slide

Design of Innovation: Innovation & Genetic Algorithms

  1. 1. THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS David E. Goldberg Department of General Engineering University of Illinois at Urbana-Champaign © 2002 David E. Goldberg, All Rights Reserved Unless otherwise noted, figures taken from The Design of Innovation: Lessons from and for Competent Genetic Algorithms , D.E. Goldberg, 2002. Reproduced with the permission of Kluwer Academic Publishers, Boston, MA. Innovation & GAs
  2. 2. Innovation This & Innovation That <ul><li>The business world is abuzz with “innovation.” </li></ul><ul><li>Popular books tell companies how to get it. </li></ul><ul><li>But little scientific understanding of what it is. </li></ul>
  3. 3. Genetic Algorithms are Coming of Age <ul><li>Genetic algorithms usage growing since 1985. </li></ul><ul><li>Used to design airplanes, compute factory schedules, even compose art and music. </li></ul><ul><li>GA millionaires have sold their companies. </li></ul><ul><li>Yet, practice of GAs remains a black art. </li></ul><ul><li>New problem requires new operators, codings, and tedious trial and error </li></ul>
  4. 4. Connect the Dots <ul><li>Wouldn’t it be nice to understand innovation scientifically? </li></ul><ul><li>GAs that solve problems once and for all? </li></ul><ul><li>Yes. Yes. Affirmative answer to both questions found in these 8 lectures. </li></ul><ul><li>Design of competent GAs, GAs that solve problems quickly, reliably, and accurately gives (1) scalable problem solver, & (2) computational theory of innovation. </li></ul>
  5. 5. Overview <ul><li>One Minute Genetic Algorithmist. </li></ul><ul><li>GAs and Innovation. </li></ul><ul><li>A lesson from the Wright brothers. </li></ul><ul><li>The goals of GA design. </li></ul><ul><li>Seven-facet theory of competent GA design. </li></ul><ul><li>A menagerie of competent GAs. </li></ul><ul><li>Four facets of GA efficiency enhancement. </li></ul><ul><li>A golden age of computational innovation? </li></ul>
  6. 6. Who Am I? <ul><li>General Engineering, University of Illinois at Urbana-Champaign, 1990-2002. </li></ul><ul><li>Engineering Mechanics, University of Alabama, Tuscaloosa, 1984-1990. </li></ul><ul><li>BS, MS, PhD in Civil Engineering (Hydraulics), 1971-75, 1976, 1980-83. </li></ul><ul><li>Project Engineer, Marketing Manager, Stoner Associates, Carlisle, PA, 1976-80. </li></ul><ul><li>Married, father of two boys, live in Champaign, IL. </li></ul>
  7. 7. Course Text <ul><li>Goldberg, D. E. (2002). The design of innovation: Lessons from and for competent genetic algorithms. Boston, MA: Kluwer Academic Publishers. </li></ul><ul><li>http://www-doi.ge.uiuc.edu/ </li></ul><ul><li>May find it useful to refer to the text for details. </li></ul>
  8. 8. Background GA Stuff <ul><li>Course is self-contained. Requires maturity of engineering, science, or math/CS BS degree. </li></ul><ul><li>Some GA background helpful, but not necessary. </li></ul><ul><li>Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley. </li></ul><ul><li>Many original papers downloadable from http:ww-illigal.ge.uiuc.edu </li></ul>www.amazon.com
  9. 9. Today’s Agenda <ul><li>One-minute genetic algorithmist. </li></ul><ul><li>The innovation intuition. </li></ul><ul><li>Intriguing applications. </li></ul>
  10. 10. The One-Minute Genetic Algorithmist <ul><li>What is a GA? </li></ul><ul><li>Solutions as chromosomes. </li></ul><ul><li>Means of evaluating fitness to purpose. </li></ul><ul><li>Create initial population. </li></ul><ul><li>Apply selection and genetic operators. </li></ul><ul><li>Repeat until solution good enough (time runs out). </li></ul>
  11. 11. What is a Genetic Algorithm? <ul><li>A genetic algorithm is a search procedure based on the mechanics of natural selection and genetics. </li></ul><ul><li>Require two things </li></ul><ul><ul><li>Survival-of-the-fittest </li></ul></ul><ul><ul><li>Variation </li></ul></ul><ul><li>Started in the cybernetics movement of the 1950s/60s. Renewed vigor in the 1980s. </li></ul>
  12. 12. Solutions as Chromosomes <ul><li>Code solutions as artificial chromosomes. </li></ul><ul><li>Possible strings = possible solutions. </li></ul><ul><li>Examples: </li></ul><ul><ul><li>Switches: Bit String: 10001 </li></ul></ul><ul><ul><li>Tour 5 cities: Permutation: 1 2 4 5 3 </li></ul></ul><ul><ul><li>Rules: If (P/E is high) then sell </li></ul></ul><ul><ul><li>Program: (- x (* x (+ x 3))) </li></ul></ul><ul><ul><li>Vector: (4.3 6.9 20.3e12) </li></ul></ul><ul><li>Manipulate code, not solutions, directly. </li></ul>
  13. 13. Determining Fitness <ul><li>Something must decide good from bad. </li></ul><ul><li>Possibilities: </li></ul><ul><ul><li>Objective function evaluates with computer </li></ul></ul><ul><ul><li>Human makes decision (subjective function) </li></ul></ul><ul><ul><li>Co-evolved against predators and prey </li></ul></ul><ul><li>Specify what wanted, not how obtained. </li></ul>
  14. 14. Operators <ul><li>Selection </li></ul><ul><li>Recombination </li></ul><ul><li>Mutation </li></ul>
  15. 15. Selection <ul><li>Darwinian survival of the fittest. </li></ul><ul><li>Give more copies to better guys. </li></ul><ul><li>Ways to do: </li></ul><ul><ul><li>roulette wheel </li></ul></ul><ul><ul><li>tournament </li></ul></ul><ul><ul><li>truncation </li></ul></ul><ul><li>By itself, pick best. </li></ul>
  16. 16. Recombination <ul><li>Combine bits and pieces of good parents. </li></ul><ul><li>Speculate on, new, possibly better children. </li></ul><ul><li>By itself, a random shuffle </li></ul><ul><li>Example, one-point X: </li></ul>Before X After X 11111 11000 00000 00111
  17. 17. Mutation <ul><li>Mutation is a random alteration of a string. </li></ul><ul><li>Change a gene, small movement in the neighborhood. </li></ul><ul><li>By itself, a random walk. </li></ul><ul><li>Example </li></ul>Before M After M 11111 11011
  18. 18. GAs & Innovation: The Fundamental Intuition <ul><li>How do individually uninteresting operators yield interesting behavior? </li></ul><ul><li>Genetic algorithm power like that of human innovation. </li></ul><ul><li>Separate selection+mutation from selection+recombination. </li></ul><ul><li>Different modes or facets of innovation. </li></ul>
  19. 19. Selection+Mutation=Improvement <ul><li>Total Quality Management: continual improvement; Japanese term: kaizen. </li></ul><ul><li>Mutation makes local changes; selection accepts the better ones. </li></ul><ul><li>A resilient and general form of hillclimbing. </li></ul><ul><li>“ Invention is nothing more than a fine deviation from, or enlargement on, a fine model.” E. Bulwer-Lytton </li></ul>
  20. 20. Selection+Recombination=Innovation <ul><li>Combine notions to form ideas. </li></ul><ul><li>“ Indeed, it is obvious that invention or discovery, be it in mathematics or anywhere else, takes place by combining ideas.” J. Hadamard </li></ul><ul><li>“ It takes two to invent anything. The one makes up combinations; the other chooses, recognizes what he wishes and what is important to him in the mass of the things which the former has imparted to him.” P. Valéry </li></ul>
  21. 21. Two Facets of Innovation <ul><li>Selection+mutation = hillclimbing or improvement. </li></ul><ul><li>Selection+crossover = cross-fertilizing innovation or simply innovation. </li></ul><ul><li>Both can be useful, especially if done right. </li></ul><ul><li>GAs as 2-edged sword: </li></ul><ul><ul><li>Technology: GAs as useful. </li></ul></ul><ul><ul><li>Science: GAs as model of innovative cognitive processes. </li></ul></ul>
  22. 22. 4 Intriguing Applications <ul><li>GE designs a Boeing 777 jet engine. </li></ul><ul><li>John Deere schedules a factory. </li></ul><ul><li>First Quadrant manages portfolio in the financial markets. </li></ul><ul><li>NMSU catches criminals. </li></ul>
  23. 23. GE Designs a Jet Engine <ul><li>GA work started in late 80s. </li></ul><ul><li>Found 2% efficiency increase w/ GA + expert system hybrid. </li></ul><ul><li>Used in 777 design. </li></ul><ul><li>GE spins off company, Engineous Systems. </li></ul>
  24. 24. A GA Schedules for John Deere <ul><li>Optimax created scheduling program. </li></ul><ul><li>Schedules planter line and 5 others. </li></ul><ul><li>Optimax bought by i2 Technologies for $60m. </li></ul><ul><li>GA technology now integrated into i2 software. </li></ul>
  25. 25. Portfolio Management with GA <ul><li>First Quadrant manages $28B with daily help from a GA. </li></ul><ul><li>Chromosomes in financial applications can be decision rules, prediction rules, or portfolio decisions. </li></ul>
  26. 26. NMSU, the Beauty & the Beast <ul><li>GA replaces criminal sketch artist. </li></ul><ul><li>NMSU system called Faceprints. </li></ul><ul><li>Work has continued on sociobiology of beauty. </li></ul><ul><li>Face at left composite of web votes + GA. </li></ul>http://www-psych.nmsu.edu/~vic/faceprints/female_study.html
  27. 27. GAs Have Special Kind of Appeal <ul><li>Going boldly where optimization has and has not gone. </li></ul><ul><li>From engineering design, to scheduling production, to financial decision making, to catching criminals and making art. </li></ul><ul><li>GAs increasingly used across spectrum of human endeavor. </li></ul><ul><li>Seem special, but is that special nature due to connection with innovation? </li></ul>
  28. 28. Summary <ul><li>One-minute genetic algorithmist. </li></ul><ul><li>The paradox of uninteresting operators. </li></ul><ul><li>The fundamental intuition of GAs: the innovation intuition. </li></ul><ul><li>Two facets of innovation: </li></ul><ul><ul><li>kaizen or continual improvement </li></ul></ul><ul><ul><li>cross-fertilizing or selectorecombinative innovation. </li></ul></ul><ul><li>Breadth of application. </li></ul>
  29. 29. Conclusions <ul><li>Innovation is important, but mysterious. </li></ul><ul><li>GAs are intriguing, but first-generation GAs unreliable. </li></ul><ul><li>Intuitively, GAs and innovation seem connected. </li></ul><ul><li>Can we design GAs that scale well on hard problems? </li></ul><ul><li>Maybe we’ll learn something about innovation along the way. Stay tuned. </li></ul>
  30. 30. THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS Produced by The Office of Continuing Engineering Education University of Illinois at Urbana-Champaign

×