Your SlideShare is downloading. ×
0
Pairwise and Problem-Specific Distance Metrics                in the Linkage Tree Genetic Algorithm                 Martin ...
Motivation       Linkage learning               Standard crossover often ineffective in presence of epistasis.             ...
Outline          1. Linkage tree genetic algorithm (LTGA).          2. Distance metrics in LTGA.          3. Experiments. ...
Linkage Tree       Linkage tree               Leaves are individual variables (string positions).               Each inter...
Linkage Tree Genetic Algorithm       LTGA procedure               Starts with a random population.               Initial p...
Learning Linkage Tree       Learning linkage tree               Start with each variable being a separate linkage group.  ...
Measuring Cluster Distances in LTGA       Distance metric based on variation of information               Distance of clus...
Pairwise Metric       Pairwise metric               Start by measuring distances between pairs of variables.              ...
Pairwise Metric       Pairwise metric               Start by measuring distances between pairs of variables.              ...
Problem-Specific Metrics       Basic idea               If we could estimate distance of clusters without computing        ...
Additively Decomposable Functions (ADFs)       Additively decomposable function               Additively decomposable func...
Problem-Specific Metric for ADFs       Distance metric for ADFs               Create graph G = (V, E).                     ...
Problem-Specific Metric for ADFs       Distance metric for ADFs               Use G to compute distances between variables ...
Experiments: Test Problems       Problems               Concatenated traps of order k.               Nearest-neighbor NK l...
Experiments: Setup       Test problem parameters, instances               Traps of order k ∈ {5, 6, 7, 8} were tested.    ...
Results: Pairwise Metric on Trap-5                                           6                                          10...
Results: Pairwise Metric on NK                                           7                        5.14                    ...
Results: Pairwise Metric on 2D Spin Glass                                           7                                     ...
Results: Problem-Specific Metric on Trap-5                                                6                                ...
Results: Problem-Specific Metric on NK                                           7                                         ...
Results: Problem-Specific Metric on 2D Spin Glass                                           8                              ...
Conclusions and Future Work       Conclusions              LTGA provides opportunities for efficiency enhancements.         ...
Acknowledgments       Acknowledgments               NSF; NSF CAREER grant ECS-0547013.               University of Missour...
Upcoming SlideShare
Loading in...5
×

Pairwise and Problem-Specific Distance Metrics in the Linkage Tree Genetic Algorithm

1,289

Published on

The linkage tree genetic algorithm (LTGA) identifies linkages between problem variables using an agglomerative hierarchical clustering algorithm and linkage trees. This enables LTGA to solve many decomposable problems that are difficult with more conventional genetic algorithms. The goal of this paper is two-fold: (1) Present a thorough empirical evaluation of LTGA on a large set of problem instances of additively decomposable problems and (2) speed up the clustering algorithm used to build the linkage trees in LTGA by using a pairwise and a problem-specific metric.

http://medal.cs.umsl.edu/files/2011001.pdf

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,289
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Pairwise and Problem-Specific Distance Metrics in the Linkage Tree Genetic Algorithm"

  1. 1. Pairwise and Problem-Specific Distance Metrics in the Linkage Tree Genetic Algorithm Martin Pelikan1 , Mark W. Hauschild1 , Dirk Thierens2 1 Missouri Estimation of Distribution Algorithms Laboratory (MEDAL) University of Missouri, St. Louis, MO pelikan@cs.umsl.edu, mwh308@umsl.edu 2 Utrecht University Utrecht, The Netherlands Dirk.Thierens@cs.uu.nl Download MEDAL Report No. 2011001 http://medal.cs.umsl.edu/files/2011001.pdfMartin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  2. 2. Motivation Linkage learning Standard crossover often ineffective in presence of epistasis. Linkage learning aims to learn interactions between problem variables to ensure that crossover does not disrupt important partial solutions and it combines them effectively. Various evolutionary algorithms capable of linkage learning exist. This study Focuses on linkage tree genetic algorithm (LTGA). Proposes and analyzes two distance metrics in LTGA. Analyzes LTGA scalability on a large number of problems.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  3. 3. Outline 1. Linkage tree genetic algorithm (LTGA). 2. Distance metrics in LTGA. 3. Experiments. 4. Summary and conclusions.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  4. 4. Linkage Tree Linkage tree Leaves are individual variables (string positions). Each internal node has two subtrees. Each node represents a subset of variables (descendants). Descendants of any node form a linkage group. Linkage groups used as masks in LTGA crossover.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  5. 5. Linkage Tree Genetic Algorithm LTGA procedure Starts with a random population. Initial population may undergo local search. Each generation performs two rounds of crossover to generate a new population of the same size. LTGA crossover Start with pair (X, Y ) of parents. For each linkage group [π1 , π2 , . . . , πk ] in T (bottom to top) Create X and Y by exchanging bits in positions {π1 , . . . , πk } between X and Y . If best(X , Y ) is better than best(X, Y ), then replace (X, Y ) with (X , Y ). The best of the two parents after applying each linkage group survives to the next population.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  6. 6. Learning Linkage Tree Learning linkage tree Start with each variable being a separate linkage group. Each step merges two closest groups. Distance of linkage groups based on variation of information. Each iteration should merge most strongly interacting groups.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  7. 7. Measuring Cluster Distances in LTGA Distance metric based on variation of information Distance of clusters Ci and Cj : H(Ci ) + H(Cj ) D(Ci , Cj ) = 2 − H(Ci , Cj ) where H(Ci , Cj ) is the entropy of Ci ∪ Cj H(Ci ) is the entropy of Ci H(Cj ) is the entropy of Cj Bottleneck in learning linkage tree Most time spent by measuring cluster distances. Can we alleviate this bottleneck? We discuss two distance metrics that address this issue.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  8. 8. Pairwise Metric Pairwise metric Start by measuring distances between pairs of variables. Cluster distance computed as average distance between pairs of variables 1 D (Ci , Cj ) = D(ci , cj ) |Ci | × |Cj | ci ∈Ci cj ∈Cj Good news We only need pairwise statistics. This results in much faster distance computation. Surprisingly, this also helps scalability.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  9. 9. Pairwise Metric Pairwise metric Start by measuring distances between pairs of variables. Cluster distance computed as average distance between pairs of variables 1 D (Ci , Cj ) = D(ci , cj ) |Ci | × |Cj | ci ∈Ci cj ∈Cj Good news We only need pairwise statistics. This results in much faster distance computation. Surprisingly, this also helps scalability.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  10. 10. Problem-Specific Metrics Basic idea If we could estimate distance of clusters without computing statistics from current population, we could possibly save lot of time in learning tree, and reduce the population sizes and number of generations. Where to get distances from? Problem-specific information. Learning from optimization runs on similar problems.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  11. 11. Additively Decomposable Functions (ADFs) Additively decomposable function Additively decomposable function: m f (X1 , . . . , Xn ) = fi (Si ) i=1 fi is ith subfunction Si is subset of variables from {X1 , . . . , Xn } Variables in located in the same subproblem are expected to interact more strongly. Can we use this fact to create a distance metric for LTGA?Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  12. 12. Problem-Specific Metric for ADFs Distance metric for ADFs Create graph G = (V, E). V = {X1 , X2 , . . . , Xn }. E = {(i, j) : Xi , Xj ∈ Sk }. Define weight of each edge from E as d(i, j) = 1. Define li,j the shortest path between i and j.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  13. 13. Problem-Specific Metric for ADFs Distance metric for ADFs Use G to compute distances between variables li,j if a path between Xi and Xj exists D (Xi , Xj ) = n otherwise Cluster distance is defined as an average of pairwise distances 1 D (Ci , Cj ) = D (ci , cj ) |Ci | × |Cj | ci ∈Ci cj ∈CjMartin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  14. 14. Experiments: Test Problems Problems Concatenated traps of order k. Nearest-neighbor NK landscapes with wrap-around neighborhoods. 2D Ising spin glass. Why these test problems? All test problems require linkage learning. All test problems are nontrivial. Yet all test problems are solvable in polynomial time.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  15. 15. Experiments: Setup Test problem parameters, instances Traps of order k ∈ {5, 6, 7, 8} were tested. NK landscapes with k = 5 were tested. For all problems, n was varied. For NK landscapes and spin glasses, for each n, 1,000 instances were generated and tested. LTGA setup Bisection was used to find minimum population size for convergence to the optimum in 10 out of 10 independent runs. For traps, bisection is repeated 10 times for each n. Max. number of generations is set to a sufficiently large value. Bit-flip local search run on initial population. Use standard, pairwise, and problem-specific metric.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  16. 16. Results: Pairwise Metric on Trap-5 6 10 1.27 Number of evaluations LTGA (original), O(n ) 1.25 LTGA (pairwise), O(n ) 5 10 4 10 2 3 10 10 Problem size, n Pairwise metric allows us to solve much larger problems. Scalability is slightly improved (surprising). Results for trap-6 and trap-7 similar.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  17. 17. Results: Pairwise Metric on NK 7 5.14 10 LTGA (original), O(n ) Number of evaluations 3.23 LTGA (pairwise), O(n ) 6 10 5 10 4 10 3 10 20 40 60 80 100 Problem size, n Pairwise metric allows us to solve much larger problems. Scalability is significantly improved (surprising).Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  18. 18. Results: Pairwise Metric on 2D Spin Glass 7 10 5.38 Number of evaluations LTGA (original), O(n ) 3.50 LTGA (pairwise), O(n ) 6 10 5 10 4 10 64 100 144 196 256 Problem size, n Pairwise metric allows us to solve much larger problems. Scalability is significantly improved (surprising).Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  19. 19. Results: Problem-Specific Metric on Trap-5 6 10 1.25 LTGA (pairwise), O(n ) Number of evaluations 1.26 LTGA (problem), O(n ) 5 10 4 10 2 3 10 10 Problem size, n Problem-specific metric similar to pairwise metric. CPU slightly decreased though with problem-specific metric.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  20. 20. Results: Problem-Specific Metric on NK 7 10 3.23 Number of evaluations LTGA (pairwise), O(n ) 2.87 6 LTGA (problem), O(n ) 10 5 10 4 10 3 10 20 40 60 80 100 Problem size, n Problem-specific metric slightly better than pairwise one. So problem-specific metric pays off.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  21. 21. Results: Problem-Specific Metric on 2D Spin Glass 8 10 4.05 Number of evaluations LTGA (problem), O(n ) 3.50 7 LTGA (pairwise), O(n ) 10 6 10 5 10 4 10 64 100 144 196 256 Problem size, n Problem-specific metric scales worse than pairwise one! Problem-specific metric is not that great for 2D spin glass.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  22. 22. Conclusions and Future Work Conclusions LTGA provides opportunities for efficiency enhancements. LTGA also provides promising tool for using problem-specific knowledge and learning from experience whensolving many instances of similar problems. Pairwise metric provides important improvement. Problem-specific metric demonstrates the ability of LTGA to exploit problem-specific knowledge on additively decomposable functions. But the results based on problem-specific information are mixed. Future work Design more robust and effective problem-specific metrics. Design methods to learn distance metrics for specific problem classes. Improve performance of LTGA on problems of complex structure. Adopt efficiency enhancement techniques for other evolutionary algorithms to LTGA, including model-directed local search, fitness modeling, parallelization, and others.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  23. 23. Acknowledgments Acknowledgments NSF; NSF CAREER grant ECS-0547013. University of Missouri; High Performance Computing Collaboratory sponsored by Information Technology Services; Research Award; Research Board.Martin Pelikan, Mark W. Hauschild, Dirk Thierens Pairwise and Problem-Specific Metrics in LTGA
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×