 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 1Lecture 12Lecture 12Hybrid intelligent system...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 2Evolutionary neural networksEvolutionary neur...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 3II Another difficulty is related to selecting...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 4y0.91345678x1x3x22-0.80.40.8-0.70.2-0.20.6-0....
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 5II The second step is to define a fitness fun...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 6II The third step is to choose the genetic op...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 7Crossover in weight optimisationCrossover in ...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 8Mutation in weight optimisationMutation in we...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 9Can genetic algorithms help us in selectingCa...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 10II The basic idea behind evolving a suitable...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 11Encoding the network architectureEncoding th...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 12Encoding of the network topologyEncoding of ...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 13The cycle of evolving a neural network topol...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 14Fuzzy evolutionary systemsFuzzy evolutionary...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 15Fuzzy partition by a 3Fuzzy partition by a 3...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 16II Black and white dots denote the training ...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 17In the rule table, each fuzzy subspace can h...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 18Fuzzy rules that correspond to theFuzzy rule...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 19To determine the rule consequent and the cer...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 20The certainty factor can be interpreted asTh...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 21II This means that patterns in a fuzzy subsp...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 22Training patterns are not necessarilyTrainin...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 23Multiple fuzzy rule tablesMultiple fuzzy rul...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 24Once the set of rulesOnce the set of rules S...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 25The number of multiple fuzzy rule tablesThe ...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 26Can we use genetic algorithms for selectingC...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 27A basic genetic algorithm for selecting fuzz...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 28The problem of selecting fuzzy rules has two...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 29The classification accuracy is more importan...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 30Step 3Step 3:: Select a pair of chromosomes ...
 Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 31Step 6Step 6:: Place the created offspring c...
Upcoming SlideShare
Loading in …5
×

2013-1 Machine Learning Lecture 07 - Michael Negnevitsky - Hybrid Intellig…

403 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
403
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

2013-1 Machine Learning Lecture 07 - Michael Negnevitsky - Hybrid Intellig…

  1. 1.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 1Lecture 12Lecture 12Hybrid intelligent systems:Hybrid intelligent systems:Evolutionary neural networks and fuzzyEvolutionary neural networks and fuzzyevolutionary systemsevolutionary systemsII IntroductionIntroductionII Evolutionary neural networksEvolutionary neural networksII Fuzzy evolutionary systemsFuzzy evolutionary systemsII SummarySummary
  2. 2.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 2Evolutionary neural networksEvolutionary neural networksII Although neural networks are used for solving aAlthough neural networks are used for solving avariety of problems, they still have somevariety of problems, they still have somelimitations.limitations.II One of the most common is associated with neuralOne of the most common is associated with neuralnetwork training. The backnetwork training. The back--propagation learningpropagation learningalgorithm cannot guarantee an optimal solution.algorithm cannot guarantee an optimal solution.In realIn real--world applications, the backworld applications, the back--propagationpropagationalgorithm might converge to a set of subalgorithm might converge to a set of sub--optimaloptimalweights from which it cannot escape. As a result,weights from which it cannot escape. As a result,the neural network is often unable to find athe neural network is often unable to find adesirable solution to a problem at hand.desirable solution to a problem at hand.
  3. 3.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 3II Another difficulty is related to selecting anAnother difficulty is related to selecting anoptimal topology for the neural network. Theoptimal topology for the neural network. The““rightright”” network architecture for a particularnetwork architecture for a particularproblem is often chosen by means of heuristics,problem is often chosen by means of heuristics,and designing a neural network topology is stilland designing a neural network topology is stillmore art than engineering.more art than engineering.II Genetic algorithms are an effective optimisationGenetic algorithms are an effective optimisationtechnique that can guide both weight optimisationtechnique that can guide both weight optimisationand topology selection.and topology selection.
  4. 4.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 4y0.91345678x1x3x22-0.80.40.8-0.70.2-0.20.6-0.3 0.1-0.20.9-0.60.10.30.5From neuron:To neuron:1 2 3 4 5 6 7 8123456780 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00.9 -0.3 -0.7 0 0 0 0 0-0.8 0.6 0.3 0 0 0 0 00.1 -0.2 0.2 0 0 0 0 00.4 0.5 0.8 0 0 0 0 00 0 0 -0.6 0.1 -0.2 0.9 0Chromosome: 0.9 -0.3 -0.7 -0.8 0.6 0.3 0.1 -0.2 0.2 0.4 0.5 0.8 -0.6 0.1 -0.2 0.9Encoding a set of weights in a chromosomeEncoding a set of weights in a chromosome
  5. 5.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 5II The second step is to define a fitness function forThe second step is to define a fitness function forevaluating the chromosomeevaluating the chromosome’’s performance. Thiss performance. Thisfunction must estimate the performance of afunction must estimate the performance of agiven neural network. We can apply here agiven neural network. We can apply here asimple function defined by the sum of squaredsimple function defined by the sum of squarederrors.errors.II The training set of examples is presented to theThe training set of examples is presented to thenetwork, and the sum of squared errors isnetwork, and the sum of squared errors iscalculated. The smaller the sum, the fitter thecalculated. The smaller the sum, the fitter thechromosome.chromosome. The genetic algorithm attemptsThe genetic algorithm attemptsto find a set of weights that minimises the sumto find a set of weights that minimises the sumof squared errors.of squared errors.
  6. 6.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 6II The third step is to choose the genetic operatorsThe third step is to choose the genetic operators ––crossover and mutation. A crossover operatorcrossover and mutation. A crossover operatortakes two parent chromosomes and creates atakes two parent chromosomes and creates asingle child with genetic material from bothsingle child with genetic material from bothparents. Each gene in the childparents. Each gene in the child’’s chromosome iss chromosome isrepresented by the corresponding gene of therepresented by the corresponding gene of therandomly selected parent.randomly selected parent.II A mutation operator selects a gene in aA mutation operator selects a gene in achromosome and adds a small random valuechromosome and adds a small random valuebetweenbetween −−1 and 1 to each weight in this gene.1 and 1 to each weight in this gene.
  7. 7.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 7Crossover in weight optimisationCrossover in weight optimisation345y6x22-0.30.9-0.70.5-0.8-0.6Parent 1x11-0.20.10.4345y6x22-0.1-0.50.2-0.90.60.3Parent 2x11 0.90.3-0.80.1 -0.7 -0.6 0.5 -0.8-0.2 0.9 0.4 -0.3 0.3 0.2 0.3 -0.9 0.60.9 -0.5 -0.8 -0.10.1 -0.7 -0.6 0.5 -0.80.9 -0.5 -0.8 0.1345y6x22-0.1-0.5-0.70.5-0.8-0.6Childx11 0.90.1-0.8
  8. 8.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 8Mutation in weight optimisationMutation in weight optimisationOriginal network345y6x22-0.30.9-0.70.5-0.8-0.6x11-0.20.10.40.1 -0.7 -0.6 0.5 -0.8-0.2 0.9345y6x220.20.9-0.70.5-0.8-0.6x11-0.20.1-0.10.1 -0.7 -0.6 0.5 -0.8-0.2 0.9Mutated network0.4 -0.3 -0.1 0.2
  9. 9.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 9Can genetic algorithms help us in selectingCan genetic algorithms help us in selectingthe network architecture?the network architecture?The architecture of the network (i.e. the number ofThe architecture of the network (i.e. the number ofneurons and their interconnections) oftenneurons and their interconnections) oftendetermines the success or failure of the application.determines the success or failure of the application.Usually the network architecture is decided by trialUsually the network architecture is decided by trialand error; there is a great need for a method ofand error; there is a great need for a method ofautomatically designing the architecture for aautomatically designing the architecture for aparticular application. Genetic algorithms mayparticular application. Genetic algorithms maywell be suited for this task.well be suited for this task.
  10. 10.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 10II The basic idea behind evolving a suitable networkThe basic idea behind evolving a suitable networkarchitecture is to conduct a genetic search in aarchitecture is to conduct a genetic search in apopulation of possible architectures.population of possible architectures.II We must first choose a method of encoding aWe must first choose a method of encoding anetworknetwork’’s architecture into a chromosome.s architecture into a chromosome.
  11. 11.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 11Encoding the network architectureEncoding the network architectureII The connection topology of a neural network canThe connection topology of a neural network canbe represented by a square connectivity matrix.be represented by a square connectivity matrix.II Each entry in the matrix defines the type ofEach entry in the matrix defines the type ofconnection from one neuron (column) to anotherconnection from one neuron (column) to another(row), where 0 means no connection and 1(row), where 0 means no connection and 1denotes connection for which the weight can bedenotes connection for which the weight can bechanged through learning.changed through learning.II To transform the connectivity matrix into aTo transform the connectivity matrix into achromosome, we need only to string the rows ofchromosome, we need only to string the rows ofthe matrix together.the matrix together.
  12. 12.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 12Encoding of the network topologyEncoding of the network topologyFrom neuron:To neuron:1 2 3 4 5 61234560 0 0 0 0 00 0 0 0 0 01 1 0 0 0 01 0 0 0 0 00 1 0 0 0 00 1 1 1 1 0345y6x22x11Chromosome:0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 1 1 1 0
  13. 13.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 13The cycle of evolving a neural network topologyThe cycle of evolving a neural network topologyNeural Network jFitness = 117Neural Network jFitness = 117Generation iTraining Data Set0 0 1.00000.1000 0.0998 0.88690.2000 0.1987 0.75510.3000 0.2955 0.61420.4000 0.3894 0.47200.5000 0.4794 0.33450.6000 0.5646 0.20600.7000 0.6442 0.08920.8000 0.7174 -0.01430.9000 0.7833 -0.10381.0000 0.8415 -0.1794Child 2Child 1CrossoverParent 1Parent 2MutationGeneration (i + 1)
  14. 14.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 14Fuzzy evolutionary systemsFuzzy evolutionary systemsII Evolutionary computation is also used in theEvolutionary computation is also used in thedesign of fuzzy systems, particularly for generatingdesign of fuzzy systems, particularly for generatingfuzzy rules and adjusting membership functions offuzzy rules and adjusting membership functions offuzzy sets.fuzzy sets.II In this section, we introduce an application ofIn this section, we introduce an application ofgenetic algorithms to select an appropriate set ofgenetic algorithms to select an appropriate set offuzzy IFfuzzy IF--THEN rules for a classification problem.THEN rules for a classification problem.II For a classification problem, a set of fuzzyFor a classification problem, a set of fuzzyIFIF--THEN rules is generated from numerical data.THEN rules is generated from numerical data.II First, we use a gridFirst, we use a grid--type fuzzy partition of an inputtype fuzzy partition of an inputspace.space.
  15. 15.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 15Fuzzy partition by a 3Fuzzy partition by a 3××××××××3 fuzzy grid3 fuzzy grid0 1A1 A2 A3X1B2B1B301X2Class 1:Class 2:µ(x1)µ(x2)010 112367459811101216151413x11x21
  16. 16.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 16II Black and white dots denote the training patternsBlack and white dots denote the training patternsofof ClassClass 1 and1 and ClassClass 2, respectively.2, respectively.II The gridThe grid--type fuzzy partition can be seen as atype fuzzy partition can be seen as arule table.rule table.II The linguistic values of inputThe linguistic values of input xx1 (1 (AA11,, AA22 andand AA33))form the horizontal axis, and the linguisticform the horizontal axis, and the linguisticvalues of inputvalues of input xx2 (2 (BB11,, BB22 andand BB33) form the) form thevertical axis.vertical axis.II At the intersection of a row and a column lies theAt the intersection of a row and a column lies therule consequent.rule consequent.Fuzzy partitionFuzzy partition
  17. 17.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 17In the rule table, each fuzzy subspace can haveIn the rule table, each fuzzy subspace can haveonly one fuzzy IFonly one fuzzy IF--THEN rule, and thus the totalTHEN rule, and thus the totalnumber of rules that can be generated in anumber of rules that can be generated in a KK××KKgrid is equal togrid is equal to KK××KK..
  18. 18.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 18Fuzzy rules that correspond to theFuzzy rules that correspond to the KK××KK fuzzyfuzzypartition can be represented in a general form as:partition can be represented in a general form as:wherewhere xxpp is a training pattern on input spaceis a training pattern on input space XX11××XX2,2,PP is the total number of training patterns,is the total number of training patterns, CCnn is theis therule consequent (eitherrule consequent (either ClassClass 1 or1 or ClassClass 2), and2), andis the certaintyis the certainty factor that a pattern in fuzzyfactor that a pattern in fuzzysubspacesubspace AAiiBBjj belongs to classbelongs to class CCnn..is Ai i = 1, 2, . . . , Kis Bj j = 1, 2, . . . , KRule Rij :IF x1pTHEN xpAND x2p∈ Cn njiCBACF xp = (x1p, x2p), p = 1, 2, . . . , PCFCFAAii BBjjCCnn
  19. 19.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 19To determine the rule consequent and the certaintyTo determine the rule consequent and the certaintyfactor, we use the following procedure:factor, we use the following procedure:Step 1Step 1:: Partition an input space intoPartition an input space into KK××KK fuzzyfuzzysubspaces, and calculate the strength of each classsubspaces, and calculate the strength of each classof training patterns in every fuzzy subspace.of training patterns in every fuzzy subspace.Each class in a given fuzzy subspace is representedEach class in a given fuzzy subspace is representedby its training patterns. The more training patterns,by its training patterns. The more training patterns,the stronger the classthe stronger the class −− in a given fuzzy subspace,in a given fuzzy subspace,the rule consequent becomes more certain whenthe rule consequent becomes more certain whenpatterns of one particular class appear more oftenpatterns of one particular class appear more oftenthan patterns of any other class.than patterns of any other class.Step 2Step 2:: Determine the rule consequent and theDetermine the rule consequent and thecertainty factor in each fuzzy subspace.certainty factor in each fuzzy subspace.
  20. 20.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 20The certainty factor can be interpreted asThe certainty factor can be interpreted asfollows:follows:II If all the training patterns in fuzzy subspaceIf all the training patterns in fuzzy subspace AAiiBBjjbelong to the same class, then the certaintybelong to the same class, then the certaintyfactor is maximum and it is certain that any newfactor is maximum and it is certain that any newpattern in this subspace will belong to this class.pattern in this subspace will belong to this class.II If, however, training patterns belong to differentIf, however, training patterns belong to differentclasses and these classes have similar strengths,classes and these classes have similar strengths,then the certainty factor is minimum and it isthen the certainty factor is minimum and it isuncertain that a new pattern will belong to anyuncertain that a new pattern will belong to anyparticular class.particular class.
  21. 21.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 21II This means that patterns in a fuzzy subspace canThis means that patterns in a fuzzy subspace canbe misclassified. Moreover, if a fuzzy subspacebe misclassified. Moreover, if a fuzzy subspacedoes not have any training patterns, we cannotdoes not have any training patterns, we cannotdetermine the rule consequent at all.determine the rule consequent at all.II If a fuzzy partition is too coarse, many patternsIf a fuzzy partition is too coarse, many patternsmay be misclassified. On the other hand, if amay be misclassified. On the other hand, if afuzzy partition is too fine, many fuzzy rulesfuzzy partition is too fine, many fuzzy rulescannot be obtained, because of the lack ofcannot be obtained, because of the lack oftraining patterns in the corresponding fuzzytraining patterns in the corresponding fuzzysubspaces.subspaces.
  22. 22.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 22Training patterns are not necessarilyTraining patterns are not necessarilydistributed evenly in the input space. As adistributed evenly in the input space. As aresult, it is often difficult to choose anresult, it is often difficult to choose anappropriate density for the fuzzy grid. Toappropriate density for the fuzzy grid. Toovercome this difficulty, we useovercome this difficulty, we use multiplemultiplefuzzy rule tablesfuzzy rule tables..
  23. 23.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 23Multiple fuzzy rule tablesMultiple fuzzy rule tablesK = 2 K = 3 K = 4 K = 5 K = 6Fuzzy IFFuzzy IF--THEN rules are generated for each fuzzyTHEN rules are generated for each fuzzysubspace of multiple fuzzy rule tables, and thus asubspace of multiple fuzzy rule tables, and thus acomplete set of rules for our case can be specifiedcomplete set of rules for our case can be specifiedas:as:2222++ 3322++ 4422++ 5522++ 6622= 90 rules.= 90 rules.
  24. 24.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 24Once the set of rulesOnce the set of rules SSALLALL is generated, a newis generated, a newpattern,pattern, xx = (= (xx1,1, xx2), can be classified by the2), can be classified by thefollowing procedure:following procedure:Step 1Step 1:: In every fuzzy subspace of the multipleIn every fuzzy subspace of the multiplefuzzy rule tables, calculate the degree offuzzy rule tables, calculate the degree ofcompatibility of a new pattern with each class.compatibility of a new pattern with each class.Step 2Step 2:: Determine the maximum degree ofDetermine the maximum degree ofcompatibility of the new pattern with each class.compatibility of the new pattern with each class.Step 3Step 3:: Determine the class with which the newDetermine the class with which the newpattern has the highest degree of compatibility,pattern has the highest degree of compatibility,and assign the pattern to this class.and assign the pattern to this class.
  25. 25.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 25The number of multiple fuzzy rule tablesThe number of multiple fuzzy rule tablesrequired for an accurate pattern classificationrequired for an accurate pattern classificationmay be large. Consequently, a complete set ofmay be large. Consequently, a complete set ofrules can be enormous. Meanwhile, these rulesrules can be enormous. Meanwhile, these ruleshave different classification abilities, and thushave different classification abilities, and thusby selecting only rules with high potential forby selecting only rules with high potential foraccurate classification, we reduce the numberaccurate classification, we reduce the numberof rules.of rules.
  26. 26.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 26Can we use genetic algorithms for selectingCan we use genetic algorithms for selectingfuzzy IFfuzzy IF--THEN rules ?THEN rules ?II The problem of selecting fuzzy IFThe problem of selecting fuzzy IF--THEN rulesTHEN rulescan be seen as a combinatorial optimisationcan be seen as a combinatorial optimisationproblem with two objectives.problem with two objectives.II The first, more important, objective is toThe first, more important, objective is tomaximise the number of correctly classifiedmaximise the number of correctly classifiedpatterns.patterns.II The second objective is to minimise the numberThe second objective is to minimise the numberof rules.of rules.II Genetic algorithms can be applied to thisGenetic algorithms can be applied to thisproblem.problem.
  27. 27.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 27A basic genetic algorithm for selecting fuzzy IFA basic genetic algorithm for selecting fuzzy IF--THEN rules includes the following steps:THEN rules includes the following steps:Step 1Step 1:: Randomly generate an initial population ofRandomly generate an initial population ofchromosomes. The population size may bechromosomes. The population size may berelatively small, say 10 or 20 chromosomes.relatively small, say 10 or 20 chromosomes.Each gene in a chromosome corresponds to aEach gene in a chromosome corresponds to aparticular fuzzy IFparticular fuzzy IF--THEN rule in the rule setTHEN rule in the rule setdefined bydefined by SSALLALL..Step 2Step 2:: Calculate the performance, or fitness, ofCalculate the performance, or fitness, ofeach individual chromosome in the currenteach individual chromosome in the currentpopulation.population.
  28. 28.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 28The problem of selecting fuzzy rules has twoThe problem of selecting fuzzy rules has twoobjectives: to maximise the accuracy of the patternobjectives: to maximise the accuracy of the patternclassification and to minimise the size of a rule set.classification and to minimise the size of a rule set.The fitness function has to accommodate both theseThe fitness function has to accommodate both theseobjectives. This can be achieved by introducing twoobjectives. This can be achieved by introducing tworespective weights,respective weights, wwPP andand wwNN, in the fitness function:, in the fitness function:wherewhere PPss is the number of patterns classifiedis the number of patterns classifiedsuccessfully,successfully, PPALLALL is the total number of patternsis the total number of patternspresented to the classification system,presented to the classification system, NNSS andand NNALLALL arearethe numbers of fuzzy IFthe numbers of fuzzy IF--THEN rules in setTHEN rules in set SS and setand setSSALLALL, respectively., respectively.ALLSNALLPNNwPPwSf s −=)(
  29. 29.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 29The classification accuracy is more important thanThe classification accuracy is more important thanthe size of a rule set. That is,the size of a rule set. That is,ALLSALL NNPPSf s −=10)(
  30. 30.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 30Step 3Step 3:: Select a pair of chromosomes for mating.Select a pair of chromosomes for mating.Parent chromosomes are selected with aParent chromosomes are selected with aprobability associated with their fitness; a betterprobability associated with their fitness; a betterfit chromosome has a higher probability of beingfit chromosome has a higher probability of beingselected.selected.Step 4Step 4:: Create a pair of offspring chromosomesCreate a pair of offspring chromosomesby applying a standard crossover operator.by applying a standard crossover operator.Parent chromosomes are crossed at the randomlyParent chromosomes are crossed at the randomlyselected crossover point.selected crossover point.Step 5Step 5:: Perform mutation on each gene of thePerform mutation on each gene of thecreated offspring. The mutation probability iscreated offspring. The mutation probability isnormally kept quite low, say 0.01. The mutationnormally kept quite low, say 0.01. The mutationis done by multiplying the gene value byis done by multiplying the gene value by ––1.1.
  31. 31.  Negnevitsky, Pearson Education, 2011Negnevitsky, Pearson Education, 2011 31Step 6Step 6:: Place the created offspring chromosomes inPlace the created offspring chromosomes inthe new population.the new population.Step 7Step 7:: RepeatRepeat Step 3Step 3 until the size of the newuntil the size of the newpopulation becomes equal to the size of the initialpopulation becomes equal to the size of the initialpopulation, and then replace the initial (parent)population, and then replace the initial (parent)population with the new (offspring) population.population with the new (offspring) population.Step 9Step 9:: Go toGo to Step 2Step 2, and repeat the process until a, and repeat the process until aspecified number of generations (typically severalspecified number of generations (typically severalhundreds) is considered.hundreds) is considered.The number of rules can be cut down to less thanThe number of rules can be cut down to less than2% of the initially generated set of rules.2% of the initially generated set of rules.

×