Discovering Beneficial Cooperative Structures for the Automated Construction of Heuristics

807 views

Published on

Talk deliverd in NICSO 2010, Granada, Spain

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
807
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Discovering Beneficial Cooperative Structures for the Automated Construction of Heuristics

  1. 1. German Terrazas [email_address] Dario Landa-Silva [email_address] Natalio Krasnogor [email_address] IV NICSO May 12 – 14, 2010 Discovering Beneficial Cooperative Structures for the Automated Construction of Heuristics Extracted from: Information Génomique et Structurale – CNRS
  2. 2. Overview <ul><li>Hyper-heuristics </li></ul><ul><li>Heuristic design </li></ul><ul><li>Our approach </li></ul><ul><li>Proof of concept </li></ul><ul><li>Methods with results </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Hyper-heuristics <ul><li>Definition: search methodologies to solve hard computational problems. </li></ul><ul><li>Aim: to manufacture novel, well performing and widely applicable heuristics </li></ul><ul><li>Research directions: </li></ul><ul><ul><li>heuristics that choose heuristics (learning assisted approaches  others’ route) </li></ul></ul><ul><ul><li>heuristics that generate heuristics (search & combine approaches  our route) </li></ul></ul><ul><li>Domain-independent problem solving strategies operating on the space of heuristics (rather than the space of solutions) </li></ul>
  4. 4. Hyper-heuristics Search methodologies choose low-level heuristics to solve hard computational problems Space of low-level heuristics Space of solutions selects & combines 120 fast & well performing Is it possible to automatically design the correct combination of low-level heuristics, the application of which results in good solutions for a given combinatorial optimisation problem ? Combinatorial Optimisation Problem HOW TO COMBINE ?
  5. 5. Q1b: How reliable are these combinations of low-level heuristics ? Q1a: Given a set of high-level heuristics (which are combinations of low-level heuristics), is it possible to generate common combinations of low-level heuristics ? Q2: What is the performance of these combinations when applied to the validation set ? Q3a: Can pattern-based heuristics be characterised by a template ? Q3b: What is the performance of the template instances when applied to the test set ? Evaluation and filtering Randomly created heuristics Patterns identification Pattern-based heuristics construction Pattern-based created heuristics Template creation Pattern-based distilling Pattern-based Heuristics Generation Cross Validation Template-based Heuristics Distilling 1 2 3 P R O B L E M  Test dataset Validation dataset Training dataset TEMPLATE FOR 
  6. 6. Proof of Concept: STSP <ul><li>Problem: </li></ul><ul><ul><li>symmetric travelling salesman problem </li></ul></ul><ul><ul><li>100 cities tour (kroA100) </li></ul></ul><ul><ul><li>Euclidean space </li></ul></ul><ul><li>Datasets: </li></ul><ul><ul><li>three independent datasets </li></ul></ul><ul><ul><li>10 TSP tours each dataset </li></ul></ul><ul><li>Low-level heuristics: </li></ul><ul><ul><li>8 TSP moves </li></ul></ul><ul><ul><li>deterministic and stochastic moves </li></ul></ul><ul><li>High-level heuristics: </li></ul><ul><ul><li>sequences of low level heuristics </li></ul></ul><ul><ul><li>operating in a pipeline fashion </li></ul></ul>2X 1CI 2X
  7. 7. 10 different training tours  10 different pattern-based heuristics Q1b: How reliable are these combinations of low-level heuristics ? 1 Information sharing Beneficial local search strategies Patterns identification Pattern-based heuristic construction Evaluation and filtering Best five heuristics kroA100_0.35612 EHHGTHHGHHTGTHHDDHDH kroA100_0.43440 ADGDADTDTHDDDCDD kroA100_0.45038 DHGHGACCCHCCACADC kroA100_0.46240 GHGHHGD kroA100_0.48562 GEGHGDD kroA100_1.46475 TGFCCGC kroA100_1.66957 AAHFFAFCFFFCGTG kroA100_2.34230 TGHGHHDHDHH kroA100_2.46724 CHCCECEHFGCFCF kroA100_2.55469 ATHGAGCDT Bottom worst heuristics A C D E T F G H CHCCECEHFGCFCF GEGHGDD TGHGHHDHDHH AAHFFAFCFFFCGTG EHHGTHHGHHTGTHHDDHDH TGFCCGC GHGHHGD ATHGAGCDT ADGDADTDTHDDDCDD DHGHGACCCHCCACADC Randomly created heuristics Q1a: Given a set of high-level heuristics, is it possible to generate common combinations of low-level heuristics ? 3OPT 2OPT 2X OROPT 1CI NI AI IO Applications of 300 randomly created heuristics Applications of GDHGHHGDCDD Vs.
  8. 8. Q2: What is the performance of these combinations when applied to the validation set ? 2 For a given PBH and across the 2 nd dataset (vkroaA100 i j where i=75, j=0,…,9) 1) 300 COPIES OF PBH ( GEGHGDD ) 2) 300 RANDOMLY GENERATED HEURISTICS 3) MAX 30% SIMILARITY 4) 10 INDEP. EVALUATIONS OF 1 AND 2 300 randomly generated heuristics 300 copies of a given PBH
  9. 9. Template Q3a: Can pattern-based heuristics be characterised by a template ? 3 Common structures Building blocks C G F E G C D GA H C G FG C T TDGA D CH D D D D A CC DHH D F CD A D CTTTC C TD F CGG GD FAED HG EF CT TEE DGA D CH H D G D CHTE CC TFC D F CD ED D GGTFF C T F AHA G A D F H T G ED CT GH DGA HGH CHD E D GAF CC HC DCD EAH D TADD C DF F G G H D TGFF HG H CTD E GA EFE CH E D D D TE C A C D DC T D HFTE D FTCG CF TG GD T HG H CT CF D H GA A CH T DD ED CC DHC D T CD DDC D FD C EEH F AT GD GDAE H E G HHGD CT GFG DGA EEAH CHD H D A C E C A D C C T D TTDGF D H C E F A GD AA HGCT AE DGA TTA CHD C D HE C D CDCD CE D FEG Distilled Heuristics
  10. 10. Q3b: What is the performance of the template instances when applied to the test dataset ? Across the 3 rd dataset (gkroaA100 i j where i=75, j=0,…,9) 1) 300 GRAMMAR GENERATED HEURISTICS 2) 300 RANDOMLY GENERATED HEURISTICS 3) MAX 30% SIMILARITY 4) 10 INDEP. EVALUATIONS OF 1 AND 2 300 randomly generated heuristics 300 grammar generated heuristics
  11. 11. Conclusions <ul><li>A general and systematic methodology for automated design of heuristics: </li></ul><ul><ul><li>Pattern-based Heuristics Generation </li></ul></ul><ul><ul><li>Cross Validation </li></ul></ul><ul><ul><li>Template-based Heuristics Distilling </li></ul></ul><ul><li>Identifies: </li></ul><ul><ul><li>information sharing mechanisms (recurrent structures) </li></ul></ul><ul><ul><li>beneficial local search strategies (recurrent structures) </li></ul></ul><ul><li>There are reliable patterns of low-level heuristics </li></ul><ul><li>Pattern-based heuristics are high performing (comparing to random) </li></ul><ul><li>It is possible to characterise pattern-based heuristics with a template </li></ul><ul><li>Template-based heuristics are high performing (comparing to random) </li></ul><ul><li>Building blocks are needed to guide search across space of solutions </li></ul><ul><li>Random LLH contribute with alternative paths for exploring space of solutions </li></ul><ul><li> Building blocks + Random LLH are orchestrated into instance of a template </li></ul><ul><li>Fine tune the MSA input parameters </li></ul><ul><li>Robust across other problems </li></ul>
  12. 12. <ul><li>Thank you. </li></ul><ul><li>EPSRC grant EP/D061571/1 Next Generation Decision Support: Automating the Heuristic Design Process </li></ul><ul><li>[1] Grammatical Rules for the Automated Construction of Heuristics . G. Terrazas and N. Krasnogor. In IEEE Congress on Evolutionary Computation. IEEE Press, 2010. (to appear) </li></ul><ul><li>[2] Discovering Beneficial Cooperative Structures for the Automated Construction of Heuristics . G. Terrazas, D. Landa-Silva and N. Krasnogor. In J.R. González et al. editors, Studies in Computational Intelligence, volume 284, NICSO 2010, pp. 89–100. Elsevier, 2010. </li></ul><ul><li>[3] Towards the Design of Heuristics by means of Self-Assembly , G. Terrazas, D. Landa-Silva and N. Krasnogor, 6th International Workshop on Developments in Computational Models, 2010. (to appear) </li></ul>German Terrazas [email_address] Dario Landa-Silva [email_address] Natalio Krasnogor [email_address] IV NICSO May 12 – 14, 2010

×