Design of Innovation: Innovation & Genetic Algorithms

Slide 1: THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS Innovation & GAs David E. Goldberg Department of General Engineering University of Illinois at Urbana-Champaign © 2002 David E. Goldberg, All Rights Reserved Unless otherwise noted, figures taken from The Design of Innovation: Lessons from and for Competent Genetic Algorithms, D.E. Goldberg, 2002. Reproduced with the permission of Kluwer Academic Publishers, Boston, MA.

Slide 2: Innovation This & Innovation That • The business world is abuzz with “innovation.” • Popular books tell companies how to get it. • But little scientific understanding of what it is. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 2

Slide 3: Genetic Algorithms are Coming of Age • Genetic algorithms usage growing since 1985. • Used to design airplanes, compute factory schedules, even compose art and music. • GA millionaires have sold their companies. • Yet, practice of GAs remains a black art. • New problem requires new operators, codings, and tedious trial and error The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 3

Slide 4: Connect the Dots • Wouldn’t it be nice to understand innovation scientifically? • GAs that solve problems once and for all? • Yes. Yes. Affirmative answer to both questions found in these 8 lectures. • Design of competent GAs, GAs that solve problems quickly, reliably, and accurately gives (1) scalable problem solver, & (2) computational theory of innovation. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 4

Slide 5: Overview • One Minute Genetic Algorithmist. • GAs and Innovation. • A lesson from the Wright brothers. • The goals of GA design. • Seven-facet theory of competent GA design. • A menagerie of competent GAs. • Four facets of GA efficiency enhancement. • A golden age of computational innovation? The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 5

Slide 6: Who Am I? • General Engineering, University of Illinois at Urbana-Champaign, 1990- 2002. • Engineering Mechanics, University of Alabama, Tuscaloosa, 1984-1990. • BS, MS, PhD in Civil Engineering (Hydraulics), 1971-75, 1976, 1980-83. • Project Engineer, Marketing Manager, Stoner Associates, Carlisle, PA, 1976- 80. • Married, father of two boys, live in Champaign, IL. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 6

Slide 7: Course Text • Goldberg, D. E. (2002). The design of innovation: Lessons from and for competent genetic algorithms. Boston, MA: Kluwer Academic Publishers. • http://www-doi.ge.uiuc.edu/ • May find it useful to refer to the text for details. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 7

Slide 8: Background GA Stuff • Course is self-contained. Requires maturity of engineering, science, or math/CS BS degree. • Some GA background helpful, but not necessary. • Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley. • Many original papers downloadable from http:\\\\www- www.amazon.com illigal.ge.uiuc.edu The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 8

Slide 9: Today’s Agenda • One-minute genetic algorithmist. • The innovation intuition. • Intriguing applications. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 9

Slide 10: The One-Minute Genetic Algorithmist • What is a GA? • Solutions as chromosomes. • Means of evaluating fitness to purpose. • Create initial population. • Apply selection and genetic operators. • Repeat until solution good enough (time runs out). The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 10

Slide 11: What is a Genetic Algorithm? • A genetic algorithm is a search procedure based on the mechanics of natural selection and genetics. • Require two things – Survival-of-the-fittest – Variation • Started in the cybernetics movement of the 1950s/60s. Renewed vigor in the 1980s. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 11

Slide 12: Solutions as Chromosomes • Code solutions as artificial chromosomes. • Possible strings = possible solutions. • Examples: – Switches: Bit String: 10001 – Tour 5 cities: Permutation: 1 2 4 5 3 – Rules: If (P/E is high) then sell – Program: (- x (* x (+ x 3))) – Vector: (4.3 6.9 20.3e12) • Manipulate code, not solutions, directly. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 12

Slide 13: Determining Fitness • Something must decide good from bad. • Possibilities: – Objective function evaluates with computer – Human makes decision (subjective function) – Co-evolved against predators and prey • Specify what wanted, not how obtained. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 13

Slide 15: Selection • Darwinian survival of the Example: Roulette Wheel fittest. Selection • Give more copies to better guys. String 1 String 2 • Ways to do: String 3 – roulette wheel String 4 – tournament – truncation • By itself, pick best. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 15

Slide 16: Recombination • Example, one-point X: • Combine bits and pieces of good parents. • Speculate on, new, Before X After X possibly better children. 11111 11000 00000 00111 • By itself, a random shuffle The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 16

Slide 17: Mutation • Example • Mutation is a random alteration of a string. • Change a gene, small movement in the Before M After M neighborhood. 11111 11011 • By itself, a random walk. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 17

Slide 18: GAs & Innovation: The Fundamental Intuition • How do individually uninteresting operators yield interesting behavior? • Genetic algorithm power like that of human innovation. • Separate selection+mutation from selection+recombination. • Different modes or facets of innovation. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 18

Slide 19: Selection+Mutation=Improvement • Total Quality Management: continual improvement; Japanese term: kaizen. • Mutation makes local changes; selection accepts the better ones. • A resilient and general form of hillclimbing. • “Invention is nothing more than a fine deviation from, or enlargement on, a fine model.” E. Bulwer-Lytton The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 19

Slide 20: Selection+Recombination=Innovation • Combine notions to form ideas. • “Indeed, it is obvious that invention or discovery, be it in mathematics or anywhere else, takes place by combining ideas.” J. Hadamard • “It takes two to invent anything. The one makes up combinations; the other chooses, recognizes what he wishes and what is important to him in the mass of the things which the former has imparted to him.” P. Valéry The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 20

Slide 21: Two Facets of Innovation • Selection+mutation = hillclimbing or improvement. • Selection+crossover = cross-fertilizing innovation or simply innovation. • Both can be useful, especially if done right. • GAs as 2-edged sword: – Technology: GAs as useful. – Science: GAs as model of innovative cognitive processes. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 21

Slide 22: 4 Intriguing Applications • GE designs a Boeing 777 jet engine. • John Deere schedules a factory. • First Quadrant manages portfolio in the financial markets. • NMSU catches criminals. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 22

Slide 23: GE Designs a Jet Engine • GA work started in late 80s. • Found 2% efficiency increase w/ GA + expert system hybrid. • Used in 777 design. • GE spins off company, Engineous Systems. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 23

Slide 24: A GA Schedules for John Deere • Optimax created scheduling program. • Schedules planter line and 5 others. • Optimax bought by i2 Technologies for $60m. • GA technology now integrated into i2 software. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 24

Slide 25: Portfolio Management with GA • First Quadrant manages $28B with daily help from a GA. • Chromosomes in financial applications can be decision rules, prediction rules, or portfolio decisions. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 25

Slide 26: NMSU, the Beauty & the Beast • GA replaces criminal sketch artist. • NMSU system called Faceprints. • Work has continued on sociobiology of beauty. • Face at left composite of web votes + GA. http://www-psych.nmsu.edu/~vic/faceprints/female_study.html The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 26

Slide 27: GAs Have Special Kind of Appeal • Going boldly where optimization has and has not gone. • From engineering design, to scheduling production, to financial decision making, to catching criminals and making art. • GAs increasingly used across spectrum of human endeavor. • Seem special, but is that special nature due to connection with innovation? The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 27

Slide 28: Summary • One-minute genetic algorithmist. • The paradox of uninteresting operators. • The fundamental intuition of GAs: the innovation intuition. • Two facets of innovation: – kaizen or continual improvement – cross-fertilizing or selectorecombinative innovation. • Breadth of application. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 28

Slide 29: Conclusions • Innovation is important, but mysterious. • GAs are intriguing, but first-generation GAs unreliable. • Intuitively, GAs and innovation seem connected. • Can we design GAs that scale well on hard problems? • Maybe we’ll learn something about innovation along the way. Stay tuned. The Design of Innovation © 2002 David E. Goldberg, University of Illinois at Urbana-Champaign 29

Slide 30: THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS Produced by The Office of Continuing Engineering Education University of Illinois at Urbana-Champaign

Slideshare.net (beta)