Investigations on Local Search based Hybrid Metaheuristics

1. INVESTIGATIONS ON LOCAL SEARCH BASED HYBRID METAHEURISTICS Habilitationskolloquium Vienna, October 1, 2014 Luca Di Gaspero Institute of Computer Graphics and Algorithms

2. THE RESEARCH AREA: PRACTICAL METHODS FOR COMBINATORIAL SEARCH AND OPTIMIZATION PROBLEMS

3. COMBINATORIAL SEARCH AND OPTIMIZATION PROBLEMS ▶ Combinatorial Search problem: find 푧̄푠.푡. 푧̄∈ ℱ ▶ Combinatorial Optimization problem: min 푧̄ 퐹(푧)̄푠.푡. 푧̄∈ ℱ 3

4. HYBRID METAHEURISTICS

5. HEURISTICS AND METAHEURISTICS ▶ Heuristics (Pearl 1984; Pólya 1945) a practical way to “tame” NP-[complete|hard] search and optimization problems ▶ non-systematic solution search or construction guided by experiential problem knowledge ▶ in general, with no guarantee (efficiency vs completeness) ▶ Metaheuristics a class of general-purpose heuristic methods ▶ independent (to some extent) from the specific problem (rely on an abstract/indirect problem representation) ▶ Local Search methods, Genetic and Evolutionary Algorithms, Ant Colony Optimization, Particle Swarm Optimization, … ▶ date back to the 1950s (Barricelli 1954; Robbins and Monro 1951), term introduced by Glover in 1986 ▶ genuinely interdisciplinary research area: AI ∩ OR ∩ CS ▶ impressive success records (state-of-the-art for Routing, Timetabling, Scheduling, SAT) ▶ empirical algorithmics (Johnson 2002; McGeoch 2012) 5

8. LOCAL SEARCH METAHEURISTICS ▶ Local Search is based the following scheme: procedure LocalSearch(푆, 푁, 퐹 ) 푠0 ← GenerateInitialSolution() while ¬StopSearch(푠푖, 푖) do 푚 ← SelectMove(푠푖, 퐹 , 푁) if AcceptableMove(푚, 푠푖, 퐹 ) then 푠푖+1 ← 푠푖 ⊕ 푚 else 푠푖+1 ← 푠푖 end if 푖 ← 푖 + 1 end while end procedure ▶ the problem interface is provided by a search space 푆, a neighborhood relation 푁 and a cost function 퐹 ▶ Local search methods differ in the following components: ▶ SelectMove: How to explore the neighborhood ▶ AcceptableMove: When to perform the move ▶ StopSearch: When to stop the search (e.g., Hill Climbing, Simulated Annealing, Tabu Search) 6

11. HYBRIDIZATION ▶ Hybridization of solution methods for NP-[complete|hard] problems is a promising research trend ▶ a skillful combination of components can lead to more effective solution methods ▶ merging complementary strengths of different solution paradigm (milden the weaknesses) ▶ Hybrid Metaheuristics a combination of one metaheuristic with components from other metaheuristics or with techniques from AI and OR (Blum et al. 2008) 7

12. HYBRIDIZATION ▶ Hybridization of solution methods for NP-[complete|hard] problems is a promising research trend ▶ a skillful combination of components can lead to more effective solution methods ▶ merging complementary strengths of different solution paradigm (milden the weaknesses) ▶ Hybrid Metaheuristics a combination of one metaheuristic with components from other metaheuristics or with techniques from AI and OR (Blum et al. 2008) 7

13. HYBRID METAHEURISTICS CLASSIFICATION (Puchinger and Raidl 2005; Raidl 2006) Hybrid Metaheuristic Metaheuristics high-level (weak coupling) level of hybridization low-level control strategy (strong coupling) batch interleaved collaborative exact techniques space decomposition homogeneity integrative order of execution parallel hybridization components other OR/AI techniques other heuristic methods problem-specific component 8

18. MOTIVATIONS AND PAPER SELECTION

19. MOTIVATIONS ▶ a skillful combination of components can lead to more effective solution methods ▶ merging complementary strengths of different solution paradigm (milden the weaknesses) ▶ Local Search is my area of expertise since my PhD ▶ contribute the algorithmic ideas also in form of software systems: ▶ EasyLocal++ (Di Gaspero and Schaerf 2003) enhancements ▶ GELATO (Cipriano, Di Gaspero, and Dovier 2013) ▶ GECODE LNS 10

22. PAPER SELECTION GUIDELINES 1. full coverage of the different kinds of hybridization considered in my work 2. span most the different categories reported in the Hybrid Metaheuristics classification 3. approaches that reached new state-of-the-art results for the problem tackled 11

25. PAPER SELECTION GUIDELINES 1. full coverage of the different kinds of hybridization considered in my work 2. span most the different categories reported in the Hybrid Metaheuristics classification 3. approaches that reached new state-of-the-art results for the problem tackled ▶ a uniform temporal coverage was not possible because of recent consolidation ←−−1 ←−−2 3−−→ ←−−4 5−−→ 6−−→ 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Figure: Journal papers, Papers in edited books, Conference papers 11

26. SELECTED PAPERS 1. L. Di Gaspero and A. Schaerf. A composite-neighborhood tabu search approach to the traveling tournament problem. Journal of Heuristics, 13(2):189–207, 2007. 2. M. Chiarandini, L. Di Gaspero, S. Gualandi, and A. Schaerf. The balanced academic curriculum problem revisited. Journal of Heuristics, 18(1):119—148, 2012. 3. R. Bellio, L. Di Gaspero, and A. Schaerf. Design and statistical analysis of a hybrid local search algorithm for course timetabling. Journal of Scheduling, 15(1):49—61, 2012. 4. L. Di Gaspero, G. Di Tollo, A. Roli, and A. Schaerf. Hybrid metaheuristics for constrained portfolio selection problem. Quantitative Finance, 11(10):1473-1488, 2011. 5. R. Cipriano, L. Di Gaspero, and A. Dovier. GELATO: a multi-paradigm tool for Large Neighborhood Search. In E.-G. Talbi (ed.). Hybrid metaheuristics. volume 434 of Studies in Computational Intelligence, pages 389-414. Springer Verlag, 2012. 6. L. Di Gaspero, A. Rendl, and T. Urli. Constraint-based approaches for Balancing Bike Sharing Systems. In C. Schulte (ed.). Principles and Practice of Constraint Programming - 19th International Conference, CP 2013, Uppsala, Sweden, September 16-20, 2013. Proceedings. volume 8124 of Lecture Notes in Computer Science, pages 758—773. Springer Verlag, 2013. ←−−1 ←−−2 3−−→ ←−−4 5−−→ 6−−→ 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Figure: Journal papers, Papers in edited books, Conference papers 12

27. OUTLINE ▶ Local Search self-hybridization ▶ Hybridization with Exact Methods 13

28. LOCAL SEARCH SELF-HYBRIDIZATION

29. NEIGHBORHOOD PORTFOLIO ▶ For a given Local Search problem encoding there might be many natural neighborhoods ▶ Neighborhood Portfolios: LS handles multiple neighborhoods procedure LocalSearch(푆, 푁1, 푁2, … , 푁푘, 퐹 ) 푠0 ← GenerateInitialSolution() while ¬StopSearch(푠푖, 푖) do 푚 ← SelectMove(푠푖, 퐹 ,푁1, 푁2, … , 푁푘) if AcceptableMove(푚, 푠푖, 퐹 ) then 푠푖+1 ← 푠푖 ⊕ 푚 else 푠푖+1 ← 푠푖 end if 푖 ← 푖 + 1 end while end procedure ▶ Operators: union 푁푖 ∪ 푁푗, sequence 푁푖 ⊗ 푁푗, composition 푁푖 ⊙ 푁푗 = (푁푖 ∪ 푁푗) ⊗ (푁푖 ∪ 푁푗) 15

32. NEIGHBORHOOD PORTFOLIO Hybrid Metaheuristics Classification Hybrid Metaheuristic Metaheuristics high-level (weak coupling) level of hybridization low-level control strategy (strong coupling) batch interleaved collaborative exact techniques space decomposition homogeneity integrative order of execution parallel hybridization components other OR/AI techniques other heuristic methods problem-specific component 16

33. THE TRAVELING TOURNAMENT PROBLEM A Neighborhood Portfolio Approach ▶ A combined Sport Scheduling problem ▶ Find a Round-Robing tournament scheduling ▶ Minimizing the travel distances (Traveling Salesman) ▶ with side constraints ▶ Highly combinatorial: non-isomorphic tournament patterns |푃푛| for 푛 teams: |푃8| = 6, |푃12| = 526, 915, 620, |푃14| = 1, 132, 835, 421, 602, 062, 347 17

34. SOLUTION REPRESENTATION FOR THE TTP Team 0 1 2 3 4 ATL FLA NYM PIT @PHI @MON NYM @PIT @ATL @FLA MON FLA PHI @MON FLA MON ATL @PIT MON PHI @PIT @PHI @NYM ATL FLA @ATL @PHI NYM PIT @NYM PIT NYM MON @ATL @FLA PHI Team 5 6 7 8 9 Distance ATL @PIT PHI MON @NYM @FLA 4414 NYM @PHI @MON PIT ATL PHI 3328 PHI NYM @ATL @FLA PIT @NYM 3724 MON FLA NYM @ATL @FLA PIT 3996 FLA @MON @PIT PHI MON ATL 5135 PIT ATL FLA @NYM @PHI @MON 3319 18

35. NEIGHBORHOOD STRUCTURES FOR THE TTP ▶ 푇 is the set {0, … , 푛 − 1} of teams, and 푅 the set {0, … , 푟 − 1} of rounds 푁1, SwapHomes: ⟨푡1, 푡2⟩, swaps home/away position of the two games between 푡1 and 푡2. 푁2, SwapTeams: ⟨푡1, 푡2⟩, swaps 푡1 and 푡2 throughout the whole solution 푁3, SwapRounds: ⟨푟1, 푟2⟩, all matches assigned to 푟1 are moved to 푟2, and vice versa 푁4, SwapMatches: ⟨푟, 푡1, 푡2⟩ the opponents of 푡1 and 푡2 in 푟 are exchanged 푁5: SwapMatchRound ⟨푡, 푟1, 푟2⟩, the opponents of 푡 in rounds 푟1 and 푟2 are exchanged ▶ as for 푁4 and 푁5, the execution breaks the tournament, therefore a repair chain is needed 19

36. TTP NEIGHBORHOOD ANALYSIS AND SELECTION ▶ In (Di Gaspero and Schaerf 2007) we perform a thorough analysis of the neighborhoods, in term of the search space connectedness ▶ the outcome of the analysis was the selection of the following composed neighborhoods ▶ 퐶푁1 = ⋃5 푖=1 푁푖 ▶ 퐶푁2 = 푁1 ∪ 푁4 ∪ 푁5 ▶ 퐶푁3 = 푁1 ∪ 푁≤4 4 ∪ 푁≤6 5 ▶ the selected neighborhoods equipped a Tabu Search (Glover and Laguna 1997) metaheuristic 20

37. EXPERIMENTAL RESULTS CN1 CN2 CN3 CN4 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 Figure: Distribution of the ranks CN1 CN2 CN3 CN4 0.00 0.05 0.10 0.15 Figure: Distribution of the normalized costs 21

38. COMPARISON WITH BEST RESULTS (at the time of writing) Instance Our Best By NL10 59,583 59,436 [1] NL12 111,483 111,248 [2] NL14 190,174 189,766 [2] NL16 270,063 267,194 [2] CIRC10 242 242 [1] CIRC12 426 408 [3] CIRC14 668 654 [3] CIRC16 1,004 928 [3] CIRC18 1,412 1,306 [4] CIRC20 1,946 1,842 [3] Instance Our Best By CON10 ∗124 — — CON12 ∗181 — — CON14 ∗252 — — CON16 328 ∗327 [5] CON18 418 — — CON20 521 — — CON22 632 — — CON24 757 — — BRA24 530,043 503,158 [4] [1] Langford, [2] Anagnostopoulos et al. 2005, [3] Lim, Rodrigues, and Zhang 2005, [4] Araùjo et al. 2005, [5] Rasmussen and Trick 2005 ∗ optimality proven by Rasmussen and Trick 2005 22

39. GENERALIZED LOCAL SEARCH MACHINES (H.H. Hoos 1999; Hoos and Stützle 2004) ▶ GLSM is a formal framework for representing Local Search methods: it is essentially a (rich) automaton ▶ states are the basic search components ▶ transitions model the search control ▶ a transition takes place when the search component has finished its execution and it is labeled with a pair 푐/푎 ▶ 푐: the condition needed to select the transition ▶ 푎: the action performed on GLSM variables • C1 C2 C3 C4 • −/c"0 −/c"c+1 (c <! )/− c #! /− −/c"0 T >" /− 23

40. GENERALIZED LOCAL SEARCH MACHINES Hybrid Metaheuristics Classification Hybrid Metaheuristic Metaheuristics high-level (weak coupling) level of hybridization low-level control strategy (strong coupling) batch interleaved collaborative exact techniques space decomposition homogeneity integrative order of execution parallel hybridization components other OR/AI techniques other heuristic methods problem-specific component 24

41. THE GENERALIZED BALANCED ACADEMIC CURRICULUM PROBLEM ▶ Basic features of the BACP: ▶ Courses, with number of credits ▶ Periods, the planning horizon divided in years and terms ▶ Load limits, i.e., min and max credits per period and min and max courses per period ▶ Precedences between courses ▶ Assign periods to courses satisfying load limits and precedences ▶ Generalization (GBACP): ▶ Curricula, subgroups of courses (students' selection) ▶ Preferences, undesired terms for courses ▶ minimize discrepancy from average load per curriculum, minimize courses in undesired terms 25

42. AN EXAMPLE OF GBACP Term 1 Term 2 Year 1 Year 2 Year 3 Curricula: 푄1 = {CS1, CS3, DB1, SE1}, 푄2 = {CS1, CS2, DB2, SE2} Preferences: CS3 not in term 2 Course Credits CS1 7 CS2 6 CS3 5 DB1 7 DB2 5 SE1 4 SE2 5 … … Precedences CS1 ≺ CS2 CS2 ≺ CS3 CS1 ≺ DB1 … ≺ … 26

43. AN EXAMPLE OF GBACP Term 1 Term 2 Year 1 CS1 CS2 (7) (6) Year 2 CS3 DB1 (5) (7) Year 3 DB2, SE1 SE2 (9) (5) Curricula: 푄1 = {CS1, CS3, DB1, SE1}, 푄2 = {CS1, CS2, DB2, SE2} Preferences: CS3 not in term 2 Course Credits CS1 7 CS2 6 CS3 5 DB1 7 DB2 5 SE1 4 SE2 5 … … Precedences CS1 ≺ CS2 CS2 ≺ CS3 CS1 ≺ DB1 … ≺ … 26

44. THE GENERALIZED BALANCED ACADEMIC CURRICULUM PROBLEM Basic GLSMs Basic Gr R • • Gr 15 • • Gr R • (a) Basic T >! /− • • Gr R • (b) Multi-start T >! /− T >! /− Multi-start • Gr R • f # fbest/r"r+1 (c) Multi-run T >! /− Multi-start-multi-run f # fbest/r"r+1 f # fbest/r"r+1 curriculum csplib8 8 (4 × 2) 46 1 46 csplib10 10 (5 × 2) 42 1 42 csplib12 12 (6 × 2) 66 1 66 UD1 9 (3 × 3) 307 37 34.62 UD2 6 (2 × 3) 268 20 27.8 UD3 9 (3 × 3) 236 31 29.81 UD4 6 (2 × 3) 139 16 25.69 UD5 6 (3 × 2) 282 31 34.32 UD6 4 (2 × 2) 264 20 27.15 −/r"0 15 T >! /− −/r"0 • Gr R • f < fbest/r "0 r >"/− T >! /− (d) Multi-start-multi-run Fig. 4 The basic GLSM templates. −/r"0 Table 1 Statistics on the GBACP instances. Instance Periods Courses Curricula Courses per Courses per Prerequ. Pref. • • Gr R • (b) Multi-start • ! /− • Gr R • f < fbest/r "0 r >"/− T >! /− Multi-run (a) Basic T >! /− Gr R • (c) Multi-run • Gr −/r (d) Fig. 4 The basic GLSM templates. Table 1 Statistics on the GBACP instances. Instance Periods Courses Curricula Courses per (Years × Terms) 15 • Gr R • (a) Basic • Gr R • (b) Multi-start • Gr R • (c) Multi-run • Gr R • f < fbest/r "0 r >"/− T >! /− (d) Multi-start-multi-run Fig. 4 The basic GLSM templates. Table 1 Statistics on the GBACP instances. Instance Periods Courses Curricula Courses per Courses per Prerequ. Pref. 27

45. r $!/f < fbest/r THE GENERALIZED BALANCED ACADEMIC CURRICULUM PROBLEM Composite GLSMs (with a kicker) 푅1 ▷ 퐾 • >" /− • f < fbest/r"0 Gr R1 f # fbest/r "r+1 K −/r"0 • Gr • −/r"0 r $!/r "r+1 f < fbest/r"0 • T >" /− −/r"0 r >!/− T >" /− 푅1 ▷ 퐾+ (b) R1 !K • >" /− >" /− fbest/r "0 • Gr R1 R2 −/r"0 • Gr • −/r"0 r $!/r "r+1 f < fbest/r"0 T >" /− r >!/− T >" /− (d) R1 !R2 16 • Gr R • Gr R • r >!/− T >" /− f < fbest/r"0 r >!/− T >" /− (a) R1 −/r"0 r >!/− r"r+1 f # fbest %r $!/ • Gr T >" /− R1 f < fbest/r "0 K r >!/− T >" /− K >!/− T >" /− f < fbest/r"0 (b) R1 !K 푅1 ▷ −/푅r"0 2 • Gr • (a) R1 R1 −/r"0 r $!/r "r+1 f < fbest/r"0 T >" /− K • (c) R1 !K+ r >!/− T >" /− (b) R1 !K • Gr R1 K • −/r"0 r"r+1 f # fbest %r $!/ T >" /− r >!/− T >" /− Fig. 5 The composite GLSM templates (b)-(f) obtained from the multi-start-multi-run machine (a). f < fbest/r "0 (c) R1 !K+ r $!/r"r+1 • Gr T >" /− T >" /− R1 r >!/− T >" /− R2 R1 f < fbest/r"0 r >!/− T >" /− R2 r $!/r "r+1 (d) R1 !R2 푅1 ▷ 푅2 ▷ 퐾+ −/r"0 • Gr • R1 f < fbest/r"0 −/r"0 r $!/r "r+1 f < fbest/r"0 T >" /− R2 K • (e) R1 !R2 !K r >!/− T >" /− (d) R1 !R2 R1 −/r"0 T >" /− R1 −/r"0 T >" /− • Gr R • f < fbest/r"0 r >!/− T >" /− (a) R1 K (b) R1 !K • Gr R1 K • −/r"0 r"r+1 f # fbest %r $!/ T >" /− r >!/− T >" /− f < fbest/r "0 (c) R1 !K+ • Gr R1 R2 T >" /− • • −/r"0 r $!/r "r+1 f < fbest/r"0 T >" /− r >!/− T >" /− (d) R1 !R2 • Gr R1 R2 K • −/r"0 r $!/r"r+1 f < fbest/r"0 T >" /− T >" /− r >!/− T >" /− (e) R1 !R2 !K • Gr R1 R1 R2 R2 r >!/− T >" /− K K T >" /− • • f # fbest %r $!/ −/r"0 f # fbest %r $!/ r"r+1 r"r+1 T >" /− T >" /− T >" /− f < fbest/r"0 r >!/− T >" /− (f) R1 !R2 !K+ f < fbest/r"0 (f) R1 !R2 !K+ Fig. 5 The composite GLSM templates (b)-(f) obtained from the multi-start-multi-run machine (a). 28

46. ALGORITHM SETTING ▶ The GLSM where equipped with a Simulated Annealing and a Dynamic Tabu Search runner ▶ DTS features an adaptive weighting strategy for the cost function ▶ the neighborhood considered are MoveCourse (MC): move one course from its period to another one SwapCourses (SC): take two courses in different periods and swap their periods ▶ As for the kicker we consider the intensification kicker relying on the moves MC, SC and MC⊗SC. 29

47. CONFIGURATIONS SELECTION ▶ We first tune each algorithm in isolation ▶ Afterwards we equip the GLSMs with the best tuning found in the previous phase ▶ Kickers have been tuned in conjunction with both SA and DTS multi-run multi-start machines. 30

48. BEST RESULTS Comparison with an exact MILP model Table: Solutions within a timeout of 320s MILP 푆퐴 ▷ 퐾+(푀퐶 ⊗ 푆퐶)푏 Inst. LB LB2 cost (퐹 ) (gap) best (gap) median (gap) UD1 10.54 111 952 (192.7) 276 (57.6) 284.5 (60.1) UD2 11.40 130 177 (16.7) 148 (6.7) 156.0 (9.5) UD3 10.30 106 651 (147.7) 163 (24.0) 171.5 (27.2) UD4 18.00 324 396 (10.6) 396 (10.5) 396.0 (10.5) UD5 11.50 3 771 (141.5) 219 (28.6) 230.0 (31.8) UD6 5.92 35 52 (21.8) 55 (25.3) 62.5 (33.6) UD7 10.95 120 -- (---) 214 (33.5) 240.0 (41.4) UD8 5.20 27 64 (53.9) 46 (30.5) 53.0 (40.1) UD9 11.66 136 1895 (273.3) 223 (28.0) 234.0 (31.1) UD10 6.16 38 63 (28.9) 48 (12.3) 54.50 (19.7) 31

49. BEST RESULTS Comparison with an exact MILP model Table: Solutions within a timeout of 3600s MILP 푆퐴 ▷ 퐾+(푀퐶 ⊗ 푆퐶)푏 Inst. LB LB2 cost (퐹 ) (gap) best (gap) median (gap) UD1 10.54 111 366 (81.5) 267 (55.0) 271.5 (56.3) UD2 11.40 130 164 (12.3) 148 (6.7) 149.0 (7.1) UD3 10.30 106 214 (42.0) 161 (23.2) 164.0 (24.3) UD4 18.00 324 396 (10.5) 396 (10.5) 396.0 (10.5) UD5 11.50 3 492 (92.8) 213 (26.9) 217.0 (28.1) UD6 5.92 35 52 (21.8) 55 (25.3) 58.0 (28.6) UD7 10.95 120 634 (129.8) 218 (34.8) 223.0 (36.4) UD8 5.20 27 64 (53.9) 42 (24.6) 46.5 (31.1) UD9 11.66 136 862 (151.7) 210 (24.3) 218.0 (26.6) UD10 6.16 38 48 (12.3) 47 (11.3) 49.5 (14.2) 31

50. HYBRIDIZATION WITH EXACT METHODS

51. Problem Decomposition

52. PROBLEM DECOMPOSITION Bi-level problem ▶ If the optimization problem can be decomposed as: min 푥̄,푦̄ 푓(푥,̄푦)̄ 푦̄∈ ℱ ⊆ ℤ푚 푥̄∈ 풢(푦)̄⊆ ℝ푛 ▶ then, the 푦̄and 푥̄variables can be dealt with different methods ▶ a metaheuristic can be more adequate for the discrete part ▶ ad-hoc methods could be applied to the continuous part ▶ once the 푦̄variables are set to 푦∗̄, the problem reduces to min 푥̄ 푓′(푥)̄ 푥̄∈ 풢(푦∗̄) ⊆ ℝ푛 where 푓′(푥)̄= 푓(푥,̄푦∗̄) 34

53. HYBRIDIZATION BY PROBLEM DECOMPOSITION Hybrid Metaheuristics Classification Hybrid Metaheuristic Metaheuristics high-level (weak coupling) level of hybridization low-level control strategy (strong coupling) batch interleaved collaborative exact techniques space decomposition homogeneity integrative order of execution parallel hybridization components other OR/AI techniques other heuristic methods problem-specific component 35

54. THE PORTFOLIO SELECTION PROBLEM Unconstrained ▶ The classical model of investments (Markowitz 1952): ▶ a set of 푛 assets 푎1, … , 푎푛 ▶ each asset 푎푖 has associated a return 푟푖 and a variance 휎푖 ▶ for each pair of assets (푎푖, 푎푗) we know the covariance 휎푖푗 of the two assets ▶ a total amount of money (normalized to 1) ▶ A portfolio is a vector of real values 푋 = {푥1, … , 푥푛}, such that: ▶ 푥푖 represents the fraction of money invested in the asset 푎푖 ▶ 푛Σ 푖=1 푟푖푥푖 is the return (or gain) to be maximized ▶ 푛Σ 푖=1 푛Σ 푗=1 휎푖푗푥푖푥푗 is the variance → risk to be minimized 36

55. THE PORTFOLIO SELECTION PROBLEM Constrained 푓(푋) = 푛Σ 푖=1 푛Σ 푗=1 휎푖푗푥푖푥푗 푠.푡. 푛Σ 푖=1 푟푖푥푖 ≥ 푅 푛Σ 푖=1 푥푖 = 1 0 ≤ 푥푖 ≤ 1 (푖 = 1, … , 푛) ▶ Cardinality constraint: the portfolio size is bounded 1 ≤ 푘푚푖푛 ≤ Σ푖 푧푖 ≤ 푘푚푎푥 ≤ 푛 ▶ Quantity constraint: the quantity of each asset is bounded 푥푖 = 0 ∨ 휖푖 ≤ 푥푖 ≤ 훿푖 (0 ≤ 휖푖 ≤ 훿푖 ≤ 1) 37

56. PROBLEM DECOMPOSITION ▶ Problem decomposition: ▶ Master solver: determine set 퐽 = {푖|푧푖 = 1} of assets in the solution ▶ Slave solver: find optimal assignment with the constraints: 휖푖 ≤ 푥푖 ≤ 훿푖 for 푖 ∈ 퐽 푥푖 = 0 for 푖 ∉ 퐽 ▶ Solution procedure: ▶ Master: simple Local Search on 푧 variables (First Descent and Steepest Descent) ▶ Slave: Quadratic Programming solver (positive definite problem with fewer assets) 38

57. CONSTRAINED EFFICIENT FRONTIER 0.0005 0.0010 0.0015 0.0020 0.0025 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 Risk Return UEF kmax = 2 kmax = 4 kmax = 8 kmax = 16 kmax = 32 ORLib Instance 4 39

58. COMPARISON WITH OTHER METAHEURISTICS Table: Comparison with (1) Schaerf and (2) Moral-Escudero et al. FD+QP SD+QP TS(1) GA+QP(2) Inst. min APL time min APL time min APL time min APL time 1 0.00366 1.5s 0.00321 3.1s 0.00409 251s 0.00321 415.1s 2 2.66104 9.6s 2.53139 14.1s 2.53617 531s 2.53180 552.7s 3 2.00146 10.1s 1.92146 16.1s 1.92597 583s 1.92150 886.3s 4 4.77157 11.2s 4.69371 18.8s 4.69816 713s 4.69507 1163.7s 5 0.24176 25.3s 0.20219 45.9s 0.20258 1603s 0.20198 1465.8s 40

59. COMPARISON WITH OTHER METAHEURISTICS Table: Comparison with (3) Crama and Schyns FD+QP SD+QP SA(3) Inst. APL time APL time APL time S1 0.72 (0.094) 0.3s 0.35 (0.0) 1.4s 1.13 (0.13) 3.2s S2 1.79 (0.22) 0.5s 1.48 (0.0) 3.1s 3.46 (0.17) 5.4s S3 10.50 (0.51) 10.2s 8.87 (0.003) 53.3s 16.12 (0.43) 30.1s 40

60. Large Neighborhood Search

61. LARGE NEIGHBORHOOD SEARCH ▶ Local Search with a particular ▶ neighborhood relation: destroy and repair a significant portion of a solution (controlled by an intensity parameter 푑) ▶ exploration strategy: use an exact method (e.g., Constraint Programming) for the repair phase, optimizing the subproblem (Pisinger and Ropke 2010; Shaw 1998) procedure LargeNeighborhoodSearch(Π = ⟨푋, 퐷, 퐶, 푓⟩, 푑) 푖 ← 0 푠 ← InitializeSolution(Π) while ¬StoppingCriterion(푠, 푖) do Π′ ← Destroy(푠, 푑, Π) 푛 ← Repair(Π′, 푓(푠)) if SolutionAccepted(푛) then 푠 ← 푛 end if 푖 ← 푖 + 1 end while end procedure ▶ the destroy step can be either random or based on the problem structure (i.e., decomposition) ▶ reuse of the CP/ILP model 42

64. HYBRIDIZATION BY LARGE NEIGHBORHOOD SEARCH Hybrid Metaheuristics Classification Hybrid Metaheuristic Metaheuristics high-level (weak coupling) level of hybridization low-level control strategy (strong coupling) batch interleaved collaborative exact techniques space decomposition homogeneity integrative order of execution parallel hybridization components other OR/AI techniques other heuristic methods problem-specific component 43

65. GELATO

66. GELATO ▶ Integrates Constraint Programming and Local Search in a single Large Neighborhood Search environment ▶ Inherits both the flexibility of CP and the efficiency of Local Search ▶ Different point of views: ▶ Gecode point of view: a library that adds Local Search features ▶ EasyLocal++ point of view: a library that adds Constraint Programming features ▶ Global point of view: an environment to model problems and hybrid solving strategies using high-level language and to search for solutions using Gecode and EasyLocal++ as low level solvers ▶ Reuse of the Constraint Programming model for the Large Neighborhood Search ▶ Reuse of the EasyLocal++ Local Search metaheuristics 45

70. ARCHITECTURE OVERVIEW 46

71. RESULTS ATSP and MEB problems 0 5 10 15 2000 2500 3000 3500 4000 4500 5000 ATSP − Instance 3 Time (s) FObj GECODE EL COMET GELATO 0 10 20 30 40 50 500000 1000000 1500000 2000000 2500000 MEB − Instance 20.24 Time (s) FObj GECODE EL COMET GELATO 47

72. CONCLUSIONS

73. CONCLUSIONS ▶ An overview of my research on Hybrid Local Search-based Metaheuristics ▶ Covers different hybridization methods ▶ Other studies on hybridization with different metaheuristics ▶ Two-level ACO for Haplotype Inference under pure parsimony (Benedettini, Roli, and Di Gaspero 2008) ▶ A Hybrid ACO+CP for Balancing Bicycle Sharing Systems (Di Gaspero, Rendl, and Urli 2013) ▶ Further studies on different problems ▶ A CP/LNS Approach for Multi-day Homecare Scheduling Problems (Di Gaspero and Urli 2014) ▶ Future work ▶ Working on collaborative methods ▶ Investigating better LNS decomposition approaches 49

74. CONCLUSIONS ▶ An overview of my research on Hybrid Local Search-based Metaheuristics ▶ Covers different hybridization methods ▶ Other studies on hybridization with different metaheuristics ▶ Two-level ACO for Haplotype Inference under pure parsimony (Benedettini, Roli, and Di Gaspero 2008) ▶ A Hybrid ACO+CP for Balancing Bicycle Sharing Systems (Di Gaspero, Rendl, and Urli 2013) ▶ Further studies on different problems ▶ A CP/LNS Approach for Multi-day Homecare Scheduling Problems (Di Gaspero and Urli 2014) ▶ Future work ▶ Working on collaborative methods ▶ Investigating better LNS decomposition approaches 49

Investigations on Local Search based Hybrid Metaheuristics

Recommended

Recommended

More Related Content

Similar to Investigations on Local Search based Hybrid Metaheuristics

Similar to Investigations on Local Search based Hybrid Metaheuristics (20)

Recently uploaded

Recently uploaded (20)

Investigations on Local Search based Hybrid Metaheuristics