• Save
Design of Innovation: Innovation & Genetic Algorithms
Upcoming SlideShare
Loading in...5

Design of Innovation: Innovation & Genetic Algorithms



This is the introductory module of a short course at the UIUC called The Design of Innovation: Lessons from and for Genetic Algorithms. It covers the material of the book of the same name.

This is the introductory module of a short course at the UIUC called The Design of Innovation: Lessons from and for Genetic Algorithms. It covers the material of the book of the same name.



Total Views
Views on SlideShare
Embed Views



3 Embeds 62

http://entrepreneurialengineer.blogspot.com 40
http://www.slideshare.net 21
http://ronpaulfreedom.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Design of Innovation: Innovation & Genetic Algorithms Design of Innovation: Innovation & Genetic Algorithms Presentation Transcript

  • THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS David E. Goldberg Department of General Engineering University of Illinois at Urbana-Champaign © 2002 David E. Goldberg, All Rights Reserved Unless otherwise noted, figures taken from The Design of Innovation: Lessons from and for Competent Genetic Algorithms , D.E. Goldberg, 2002. Reproduced with the permission of Kluwer Academic Publishers, Boston, MA. Innovation & GAs
  • Innovation This & Innovation That
    • The business world is abuzz with “innovation.”
    • Popular books tell companies how to get it.
    • But little scientific understanding of what it is.
  • Genetic Algorithms are Coming of Age
    • Genetic algorithms usage growing since 1985.
    • Used to design airplanes, compute factory schedules, even compose art and music.
    • GA millionaires have sold their companies.
    • Yet, practice of GAs remains a black art.
    • New problem requires new operators, codings, and tedious trial and error
  • Connect the Dots
    • Wouldn’t it be nice to understand innovation scientifically?
    • GAs that solve problems once and for all?
    • Yes. Yes. Affirmative answer to both questions found in these 8 lectures.
    • Design of competent GAs, GAs that solve problems quickly, reliably, and accurately gives (1) scalable problem solver, & (2) computational theory of innovation.
  • Overview
    • One Minute Genetic Algorithmist.
    • GAs and Innovation.
    • A lesson from the Wright brothers.
    • The goals of GA design.
    • Seven-facet theory of competent GA design.
    • A menagerie of competent GAs.
    • Four facets of GA efficiency enhancement.
    • A golden age of computational innovation?
  • Who Am I?
    • General Engineering, University of Illinois at Urbana-Champaign, 1990-2002.
    • Engineering Mechanics, University of Alabama, Tuscaloosa, 1984-1990.
    • BS, MS, PhD in Civil Engineering (Hydraulics), 1971-75, 1976, 1980-83.
    • Project Engineer, Marketing Manager, Stoner Associates, Carlisle, PA, 1976-80.
    • Married, father of two boys, live in Champaign, IL.
  • Course Text
    • Goldberg, D. E. (2002). The design of innovation: Lessons from and for competent genetic algorithms. Boston, MA: Kluwer Academic Publishers.
    • http://www-doi.ge.uiuc.edu/
    • May find it useful to refer to the text for details.
  • Background GA Stuff
    • Course is self-contained. Requires maturity of engineering, science, or math/CS BS degree.
    • Some GA background helpful, but not necessary.
    • Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley.
    • Many original papers downloadable from http:ww-illigal.ge.uiuc.edu
  • Today’s Agenda
    • One-minute genetic algorithmist.
    • The innovation intuition.
    • Intriguing applications.
  • The One-Minute Genetic Algorithmist
    • What is a GA?
    • Solutions as chromosomes.
    • Means of evaluating fitness to purpose.
    • Create initial population.
    • Apply selection and genetic operators.
    • Repeat until solution good enough (time runs out).
  • What is a Genetic Algorithm?
    • A genetic algorithm is a search procedure based on the mechanics of natural selection and genetics.
    • Require two things
      • Survival-of-the-fittest
      • Variation
    • Started in the cybernetics movement of the 1950s/60s. Renewed vigor in the 1980s.
  • Solutions as Chromosomes
    • Code solutions as artificial chromosomes.
    • Possible strings = possible solutions.
    • Examples:
      • Switches: Bit String: 10001
      • Tour 5 cities: Permutation: 1 2 4 5 3
      • Rules: If (P/E is high) then sell
      • Program: (- x (* x (+ x 3)))
      • Vector: (4.3 6.9 20.3e12)
    • Manipulate code, not solutions, directly.
  • Determining Fitness
    • Something must decide good from bad.
    • Possibilities:
      • Objective function evaluates with computer
      • Human makes decision (subjective function)
      • Co-evolved against predators and prey
    • Specify what wanted, not how obtained.
  • Operators
    • Selection
    • Recombination
    • Mutation
  • Selection
    • Darwinian survival of the fittest.
    • Give more copies to better guys.
    • Ways to do:
      • roulette wheel
      • tournament
      • truncation
    • By itself, pick best.
  • Recombination
    • Combine bits and pieces of good parents.
    • Speculate on, new, possibly better children.
    • By itself, a random shuffle
    • Example, one-point X:
    Before X After X 11111 11000 00000 00111
  • Mutation
    • Mutation is a random alteration of a string.
    • Change a gene, small movement in the neighborhood.
    • By itself, a random walk.
    • Example
    Before M After M 11111 11011
  • GAs & Innovation: The Fundamental Intuition
    • How do individually uninteresting operators yield interesting behavior?
    • Genetic algorithm power like that of human innovation.
    • Separate selection+mutation from selection+recombination.
    • Different modes or facets of innovation.
  • Selection+Mutation=Improvement
    • Total Quality Management: continual improvement; Japanese term: kaizen.
    • Mutation makes local changes; selection accepts the better ones.
    • A resilient and general form of hillclimbing.
    • “ Invention is nothing more than a fine deviation from, or enlargement on, a fine model.” E. Bulwer-Lytton
  • Selection+Recombination=Innovation
    • Combine notions to form ideas.
    • “ Indeed, it is obvious that invention or discovery, be it in mathematics or anywhere else, takes place by combining ideas.” J. Hadamard
    • “ It takes two to invent anything. The one makes up combinations; the other chooses, recognizes what he wishes and what is important to him in the mass of the things which the former has imparted to him.” P. Valéry
  • Two Facets of Innovation
    • Selection+mutation = hillclimbing or improvement.
    • Selection+crossover = cross-fertilizing innovation or simply innovation.
    • Both can be useful, especially if done right.
    • GAs as 2-edged sword:
      • Technology: GAs as useful.
      • Science: GAs as model of innovative cognitive processes.
  • 4 Intriguing Applications
    • GE designs a Boeing 777 jet engine.
    • John Deere schedules a factory.
    • First Quadrant manages portfolio in the financial markets.
    • NMSU catches criminals.
  • GE Designs a Jet Engine
    • GA work started in late 80s.
    • Found 2% efficiency increase w/ GA + expert system hybrid.
    • Used in 777 design.
    • GE spins off company, Engineous Systems.
  • A GA Schedules for John Deere
    • Optimax created scheduling program.
    • Schedules planter line and 5 others.
    • Optimax bought by i2 Technologies for $60m.
    • GA technology now integrated into i2 software.
  • Portfolio Management with GA
    • First Quadrant manages $28B with daily help from a GA.
    • Chromosomes in financial applications can be decision rules, prediction rules, or portfolio decisions.
  • NMSU, the Beauty & the Beast
    • GA replaces criminal sketch artist.
    • NMSU system called Faceprints.
    • Work has continued on sociobiology of beauty.
    • Face at left composite of web votes + GA.
  • GAs Have Special Kind of Appeal
    • Going boldly where optimization has and has not gone.
    • From engineering design, to scheduling production, to financial decision making, to catching criminals and making art.
    • GAs increasingly used across spectrum of human endeavor.
    • Seem special, but is that special nature due to connection with innovation?
  • Summary
    • One-minute genetic algorithmist.
    • The paradox of uninteresting operators.
    • The fundamental intuition of GAs: the innovation intuition.
    • Two facets of innovation:
      • kaizen or continual improvement
      • cross-fertilizing or selectorecombinative innovation.
    • Breadth of application.
  • Conclusions
    • Innovation is important, but mysterious.
    • GAs are intriguing, but first-generation GAs unreliable.
    • Intuitively, GAs and innovation seem connected.
    • Can we design GAs that scale well on hard problems?
    • Maybe we’ll learn something about innovation along the way. Stay tuned.
  • THE DESIGN OF INNOVATION: LESSONS FROM AND FOR COMPETENT GENETIC ALGORITHMS Produced by The Office of Continuing Engineering Education University of Illinois at Urbana-Champaign