• Save
Genetic algorithms
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Genetic algorithms

on

  • 1,675 views

Priti Srinivas Sajja is an Associate Professor working with Post Graduate Department of Computer Science, Sardar Patel University, India since 1994. She specializes in Artificial Intelligence ...

Priti Srinivas Sajja is an Associate Professor working with Post Graduate Department of Computer Science, Sardar Patel University, India since 1994. She specializes in Artificial Intelligence especially in knowledge-based systems, soft computing and multiagent systems. She is co-author of Knowledge-Based Systems (2009) and Intelligent Technologies for Web Applications (2012).

She has 104 publications in books, book chapters, journals, and in the proceedings of national and international conferences. Three of her publications have won best research paper awards. Visit pritisajja.info for material.

Statistics

Views

Total Views
1,675
Views on SlideShare
1,675
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
1

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Genetic algorithms Presentation Transcript

  • 1. Genetic Algorithms Priti Srinivas Sajja Associate Professor Department of Computer Science Sardar Patel UniversityVisit priti sajja.info for detail Created By Priti Srinivas Sajja 1
  • 2. Genetic AlgorithmsIntroductionIntroductionFundamentalsAlgorithmFunctionOptimization - 1FunctionOptimization - 2Ordering ProblemsEdgeRecombinationSchemaGeneticProgramming 2 Created By Priti Srinivas Sajja
  • 3. Genetic AlgorithmsIntroductionIntroduction • The basic purpose of a genetic algorithm (GA) is to mimic Nature’s evolutionary approachFundamentals • The algorithm is based on the process of natural selection—Charles Darwin’s “survival of the fittest.”AlgorithmFunction • GAs can be used in problem solving, function optimizing,Optimization - 1 machine learning, and in innovative systems.Function • Historically, GAs were first conceivedOptimization - 2 and used by John Holland (1975)Ordering Problems at the University of Michigan.Edge • Genetic algorithms are useful and efficient when:Recombination • The search space is large, complex, or poorly understoodSchema • Domain knowledge is scarce or expert knowledge is difficult to encode to narrow the search spaceGenetic • No mathematical analysis is availableProgramming Image adapted from 3 Created By Priti Srinivas Sajja http://today.mun.ca/news.php?news_id=2376
  • 4. Genetic AlgorithmsIntroductionIntroduction • According to Hales (2006), instead of just accepting a random testing strategy to arrive at a solution, one may choose aFundamentals strategy like:Algorithm • Generate a set of random solutionsFunction • RepeatOptimization - 1 • Test each solution in the set (rank them)Function • Remove any bad solutions from the set (bad solutionsOptimization - 2 refer to solutions which are less likely to provide effective answers according to fitness criteria given byOrdering Problems experts)Edge • Duplicate any good solutions or make small changesRecombination to some of them • Until an appropriate solution is achieved.SchemaGeneticProgramming 4 Created By Priti Srinivas Sajja
  • 5. Genetic AlgorithmsIntroduction • DNA is the building block of bio-cells and strings of DNA is chromosomes.FundamentalsFundamentals • Chromosomes can be bit strings, real numbers, permutations of elements, lists of rules, and dataAlgorithm structures.Function • On each DNA string, there is a set of genes responsibleOptimization - 1 for some property of a human (or living thing) to whichFunction it belongs.Optimization - 2 • Such properties, to name a few, are height, skin color,Ordering Problems hair color, and eye color. • A genotype is a collection of such genes representingEdge possible solutions in a domain space.Recombination • This space, referred to as a search space, comprises allSchema possible solutions to the problem at hand.GeneticProgramming 5 Created By Priti Srinivas Sajja
  • 6. Genetic AlgorithmsIntroduction • With every evolutionary step, known as a generation, the individuals in the current population are decodedFundamentalsFundamentals and evaluated according to some predefined quality criterion, referred to as the fitness or fitness function.Algorithm • The degree of fitness is related to the strength of theFunction genes in an individual.Optimization - 1Function • When new child is born, the parents’ genes areOptimization - 2 inherited sometimes directly, without any modification, if they are strong enough. This process isOrdering Problems known as duplication (exact resemblance of the childEdge to its parent) in new generation.Recombination • Strong genes are normally represented in the newSchema generation.GeneticProgramming 6 Created By Priti Srinivas Sajja
  • 7. Genetic AlgorithmsIntroduction • An initial population is considered to have a fixed number of individuals (also known as offspring) containing buildingFundamentals blocks or chromosomes on which genes are set.Fundamentals • A fit individual with strong genes can reproduce itself.Algorithm • Otherwise, in the next generation, the strong genes from one or more individuals can be combined, resulting in aFunction stronger individual. Over time, the individuals in theOptimization - 1 population become better adapted to their environment.FunctionOptimization - 2Ordering Problems (a) (b) (c) (d) A B Crossover Individual A Individual BEdge A Mutated 1 1 1 1Recombination 0 0 1 0 0 1 0 1 1 1 1 1SchemaGenetic Figure : Operations in the process of natural selectionProgramming 7 Created By Priti Srinivas Sajja
  • 8. Genetic AlgorithmsIntroductionFundamentals Start with initial population by randomly selected Initial population IndividualsG A CycleAlgorithm ModifyFunction with Selection Crossover Mutation operationsOptimization - 1Function Evaluate fitness of new Evaluating new individuals through fitness functionOptimization - 2 individualsOrdering Problems Update population with better individuals and Modify the population repeatEdgeRecombination Figure : Genetic cycleSchemaGeneticProgramming 8 Created By Priti Srinivas Sajja
  • 9. Genetic AlgorithmsIntroductionFundamentalsG A CycleAlgorithm The most common method of encoding is with a binary string. Each individual represents one binary string. EachFunction bit in this string can represent some characteristic of theOptimization - 1 solution. Then the individual (or chromosome) can beFunction represented as shown below:Optimization - 2Ordering ProblemsEdge Individual 1 0 1 0 1 0 1 0 0 1 0 0 1 0 1 1 1Recombination Individual 2 1 1 0 0 0 1 1 0 0 0 0 1 0 0 1 0SchemaGeneticProgramming 9 Created By Priti Srinivas Sajja
  • 10. Genetic AlgorithmsIntroduction Individual 1 0 1 0 1 0 1 0 0 1 0 0 1 0 1 1 1 NewFundamentals Individual 1 0 1 0 1 1 1 0 1 1 0 0 1 0 0 1 1 Individual 2 1 1 0 0 0 1 1 0 0 0 0 1 0 0 1 0G A CycleAlgorithm New Individual 2 0 0 0 1 0 1 1 0 0 1 0 1 0 0 1 0FunctionOptimization - 1Function Individual 1 0 1 0 1 0 0 0 0 1 0 0 1 0 1 1 1Optimization - 2 Individual 2 1 1 0 0 0 1 Crossover point0 1 0 0 0 1 0 0 1 0Ordering Problems Substrings of same lengthEdge NewRecombination Individual 1 1 1 0 0 0 0 0 0 1 0 0 1 0 1 1 1 NewSchema Individual 2 0 1 0 1 0 1 1 0 0 0 0 1 0 0 1 0GeneticProgramming 10 Created By Priti Srinivas Sajja Images from wpclipart.com
  • 11. Genetic AlgorithmsIntroductionFundamentalsG A CycleAlgorithm E F D AFunctionOptimization - 1 CFunction BOptimization - 2Ordering Problems Figure: Roulette wheel selectionEdgeRecombinationSchemaGeneticProgramming 11 Created By Priti Srinivas Sajja
  • 12. Genetic AlgorithmsIntroduction Consider a simple function f(x)=x*x on the integerFundamentals interval [0,30]. 1000Algorithm f(x) 500FunctionFunctionOptimization - 1Optimization - 1Function 0 x 30Optimization - 2 • Consider the following four strings in initial population: 01101Ordering Problems 11000Edge 01000Recombination 10011 • Use a simple genetic algorithm composed of threeSchema operators(Reproduction, Crossover and Mutation) to optimize the aforementioned function in given interval.GeneticProgramming 12 Created By Priti Srinivas Sajja
  • 13. Genetic AlgorithmsIntroduction Sr. Value of x Decimal Fitness f(x) = x*x Roulette wheel No. of value of x (Value in selection countFundamentals indivi decimal) dual 1 01101 13 169 1Algorithm 2 11000 24 576 2FunctionFunction 3 01000 08 064 0Optimization - 1Optimization - 1 4 10011 19 361 1FunctionOptimization - 2 Table : First Generation of f(x)=x*xOrdering Problems 1Edge Figure : Roulette wheel 2Recombination 3 presentation for the first 4 generation shown in theSchema f(x)=x*x exampleGeneticProgramming 13 Created By Priti Srinivas Sajja
  • 14. Genetic AlgorithmsIntroduction Sr. No. of Value of x New individual Operation Fitness f(x) = selected after mutation site x*x individual and cross-over (Value inFundamentals decimal) 1 01101 01100 5 12*12=144 2 11000 11001 5 25*25=625Algorithm 3 11000 11011 3 27*27= 729FunctionFunction 4 10011 10000 3 16*16= 256Optimization - 1Optimization - 1 Table : Mutation and crossover after reproduction for f(x)=x*xFunctionOptimization - 2 Sr. No. of Value of x Decimal value of y Fitness f(x) = x*x Individual (Value in Decimal)Ordering Problems 1 01100 12 144Edge 2 11001 25 625Recombination 3 11011 27 729 4 10000 16 256Schema Table: Second generation of the function f(x)=x*xGeneticProgramming 14 Created By Priti Srinivas Sajja
  • 15. Genetic AlgorithmsIntroduction Sr. No. of Value of x Value of y Bit string Fitness f(X) = Roulette individual x+y wheelFundamentals (Value in selection decimal) countAlgorithm 1 01011 01010 0101101010 10+11= 21 1 2 11000 01100 1100001100 24+12=36 2Function 3 01001 00101 0100100101 9+5=14 0Optimization - 1 4 01011 00111 0101100111 11+7=18 1FunctionFunctionOptimization - 2Optimization - 2 Table: First generation of f(x,y)= x+y 1Ordering Problems 2 • Maximize f(x,y) = x + y in the interval of [0, 3Edge 30] with binary encoding strategy of size 10, 4Recombination with the first 5 bits representing a value of the first variable x, and the remaining bitsSchema representing a value of the second variable y. Figure : Roulette wheel presentation for the generation shown in the f(x,y)= x+y exampleGeneticProgramming 15 Created By Priti Srinivas Sajja
  • 16. Genetic AlgorithmsIntroductionFundamentalsAlgorithmFunctionOptimization - 1FunctionFunctionOptimization - 3Optimization - 2Ordering ProblemsEdgeRecombinationSchemaGeneticProgramming 16 Created By Priti Srinivas Sajja
  • 17. Genetic AlgorithmsIntroduction Initial Population for TSPFundamentals covering any five city from given six citiesAlgorithm (5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)FunctionOptimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)FunctionOptimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)OrderingProblemsOrdering ProblemsEdge (4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)RecombinationSchema Example Courtesy: Jim Cohoon and Kimberly HanksGeneticProgramming 17 Created By Priti Srinivas Sajja
  • 18. Genetic AlgorithmsIntroductionFundamentals Generation of IndividualsAlgorithm (5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)FunctionOptimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)FunctionOptimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)OrderingProblemsOrdering Problems (4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)EdgeRecombinationSchema (3,4,5,6,2)GeneticProgramming 18 Created By Priti Srinivas Sajja
  • 19. Genetic AlgorithmsIntroductionFundamentals Generation of more IndividualsAlgorithm (5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)FunctionOptimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)FunctionOptimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)OrderingProblemsOrdering Problems (4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)EdgeRecombinationSchema (3,4,5,6,2) (5,4,2,6,3)GeneticProgramming 19 Created By Priti Srinivas Sajja
  • 20. Genetic AlgorithmsIntroductionFundamentals MutationAlgorithm (5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)FunctionOptimization - 1 (2,3,4,6,5) (4,3,6,2,5) (3,4,5,2,6)FunctionOptimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)OrderingProblemsOrdering Problems (4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)EdgeRecombinationSchema (3,4,5,6,2) (5,4,2,6,3)GeneticProgramming 20 Created By Priti Srinivas Sajja
  • 21. Genetic AlgorithmsIntroduction Elimination of the worst toursFundamentals (5,3,4,6,2) (2,4,6,3,5) (4,3,6,5,2)AlgorithmFunction (2,3,4,6,5) (2,3,6,4,5) (3,4,5,2,6)Optimization - 1FunctionOptimization - 2 (3,5,4,6,2) (4,5,3,6,2) (5,4,2,3,6)OrderingProblemsOrdering Problems (4,6,3,2,5) (3,4,2,6,5) (3,6,5,1,4)EdgeRecombinationSchema (3,4,5,6,2) (5,4,2,6,3)GeneticProgramming 21 Created By Priti Srinivas Sajja
  • 22. Genetic AlgorithmsIntroduction • Finding an optimal ordering for a sequence of N items. • For example, in a traveling salesman problem, consider thereFundamentals are four cities 1,2,3 and 4. • Each city is then, labeled by a unique bit string.Algorithm • A common fitness function for the problem is length of theFunctionOptimization - 1 candidate tour.Function • A natural way is the permutation, so that 3214 is oneOptimization - 2 candidate’s tour and 4123 is another.OrderingProblemsOrdering Problems • This representation is problematic for genetic algorithm because Mutation and crossover do not produce necessarilyEdgeRecombination leagal tours.Schema • For example, cross over between position 2 and 3 produces (in the example 3214 and 4123) produces the individuals 3223 andGenetic 4114, both of which are illegal tours.Programming 22 Created By Priti Srinivas Sajja
  • 23. Genetic AlgorithmsIntroductionFundamentals • Adopting a different representationsAlgorithmFunction • Designing a special crossover operatorsOptimization - 1FunctionOptimization - 2 • Penalizing the illegal solution with the properOrderingProblemsOrdering Problems fitness function.EdgeRecombinationSchemaGeneticProgramming 23 Created By Priti Srinivas Sajja
  • 24. Genetic AlgorithmsIntroduction 362145Fundamentals 364125 521364Algorithm Original IndividualsFunctionOptimization - 1 Key Adjacent KeysFunctionOptimization - 2 1 2, 2, 3, 4 2 1, 1, 5, 6Ordering Problems 3 1, 6, 6EdgeEdge 4 1, 5, 6RecombinationRecombination 5 2, 4 6 2, 3, 3, 4SchemaGeneticProgramming 24 Created By Priti Srinivas Sajja
  • 25. Genetic AlgorithmsIntroduction • For the problems, which are complex, noisy or dynamic, it is virtually impossible t to predict the performance of a genetic algorithm.Fundamentals • Holland[1975], introduced the notion of schema to explainAlgorithm how genetic algorithms search for region of high fitness.Function • A schema is a template, defined over alphabet{0,1,*} whichOptimization - 1 describes a pattern of bit strings in the search space {0,1}l,Function where l is length of a bit string.Optimization - 2 • For example, the two strings A and B have several bits inOrdering Problems common. We can use schemas to describe the patterns theseEdge two strings A=100111 and B=010011 share:Recombination **0*11SchemaSchema ****11 **0***Genetic **0**1Programming 25 Created By Priti Srinivas Sajja
  • 26. Genetic AlgorithmsIntroductionFundamentals • A bit string x that matches a schema S’s pattern is said to be an instance of S. For example, A and B both are instances of the schemas.Algorithm • Order of schema is the number of defined bits in it. ForFunction example, order of the first schema **0*11 is 3.Optimization - 1 • Defining length of a schema is the distance between theFunction left most and right most defined bits in the schema.Optimization - 2 • For example, the defining length of **0**1 is 3. Other examples areOrdering Problems • for S1 = [01#1#], (S1) = 4 – 1 = 3Edge • for S2 = [##1#1010], (S2) = 8 – 3 = 5RecombinationSchemaSchemaGeneticProgramming 26 Created By Priti Srinivas Sajja
  • 27. Genetic Algorithms • Schema defines hyper plane in theIntroduction search space {0,1}l . • This figure defines four hyper planesFundamentals corresponding to the four different schemas.Algorithm • Any point in the space can beFunction simultaneously an instance of twoOptimization - 1 schemas.Function • The fitness of any bit string in theOptimization - 2 population gives some information about the average fitness of the 2l differentOrdering Problems schemas (class) of which it is an instance. *0***Edge • So an explicit evaluation of population0**** ofRecombination M individual strings is also an implicit 1**** *1*** (implied) evaluation of a much larger is an instance ofSchemaSchema number of schemas. 1**** and *0***.Genetic • This is referred to as implicit parallelism.Programming 27 Created By Priti Srinivas Sajja
  • 28. Genetic Algorithms • Programs written in subset of LISP language andIntroduction • Programs which can be represented in a tree are the candidate ofFundamentals evolution under genetic fashion/natural selection. • Population of a random trees are generated and evaluated as in theAlgorithm standard genetic algorithm. • They have special crossover function.FunctionOptimization - 1 Expression: x2+3xy+y2Function LISP: (+(*xx)(*xy)(*yy))Optimization - 2Ordering Problems +EdgeRecombination * * *Schema X x 3 x y y yGeneticGeneticProgrammingProgramming 28 Created By Priti Srinivas Sajja
  • 29. Genetic AlgorithmsIntroduction • Human written programs are design elegant and generalFundamentals but the genetic generated programs are often needlessly complicated and not revealing the underplayingAlgorithm algorithm.Function • For cos2x we write (-1(*2(*sin x)(sin x))))Optimization - 1 • Genetic program discoversFunction • (sin(-(-2(*x2))(sin(sin(sin(sin(sin(sin(*(sin(sin 1)))))))))))Optimization - 2 • The evolved programs are inelegant, redundant, inefficientOrdering Problems and difficult for human read. • They give adhoc type of solutions that evolve in natureEdgeRecombination through gene duplication, mutation and modifying structures.Schema • If successful, it leads to the design revolutions.GeneticGeneticProgrammingProgramming 29 Created By Priti Srinivas Sajja
  • 30. Genetic AlgorithmsIntroduction • Optimization, combinatorial, and scheduling problemsFundamentals • Automatic programming • Game playingAlgorithm • Self-managing and sorting networks • Machine and robot learningFunction • Evolving artificial neural network (ANN), rules-based systems andOptimization - 1 other hybrid architectures of soft computingFunction • Designing and controlling robotsOptimization - 2 • Modeling natural systems (to model processes of innovation)Ordering Problems • Emergence of economic markets • Ecological phenomenaEdge • Study of evolutionary aspects of social systems, such as theRecombination evolution of cooperation, evolution of communication, and trail-Schema following behavior in ants • Artificial life models (systems that model interactions betweenGenetic species evolution and individual learning)Programming 30 Created By Priti Srinivas Sajja
  • 31. Genetic AlgorithmsIntroductionFundamentalsAlgorithmFunctionOptimization - 1FunctionOptimization - 2 References • “Knowledge-based systems”, Akerkar RA and PritiOrdering Problems Srinivas Sajja, Jones & Bartlett Publishers, Sudbury, MA, USA (2009)EdgeRecombination • Allen B. Tucker,Jr. The Computer Science and Engineering handbook, CRC Press, 1997, (pp. 557 to pp.569)Schema • Articulte.comGenetic • Pit.eduProgramming 31 Created By Priti Srinivas Sajja