A performance analysis of metaheuristics and hybrid metaheuristics for the travel salesman problem is presented. Four single classical metaheuristics (genetic algorithm, memetic algorithm, iterated local search, and simulated annealing) were used. In addition, hybrid variations using nine different heuristic techniques for the local search, the mutation, and the intensification were used. The performance analysis was made using the Friedman test, and for the simulated annealing and local search algorithms statistical evidence was found that hybridization provides a difference in performance, while no evidence was found for the genetic and memetic algorithms. Up to six combinations were found to improve performance, five of them based on local search and one more based on simulated annealing.

1. Performance Analysis of Metaheuristics and Hybrid
Metaheuristics for the Travel Salesman Problem
David A. Gutiérrez-Hernández*,1
, Marco A. Escobar**, Josué Del Valle-Hernández*, Miguel Gómez-Díaz*,
José Luis Villanueva-Rodríguez*, Juan M. López-López***, Claudia Lara-Rendón*,
Rossana Rodríguez-Montero*, Héctor Nava-Martínez****
*Tecnológico Nacional de México, Instituto Tecnológico de León, León, Guanajuato, 37290, México
**Universidad de La Salle Bajío campus Salamanca, Salamanca, Guanajuato, 36700, México
***Escuela Colombiana de Ingeniería Julio Garavito, Bogotá, Colombia
****Centro de Innovación Aplicada en Tecnologías Competitivas; León, Guanajuato, 37545, México
1
david.gutierrez@itleon.edu.mx
Abstract– A performance analysis of metaheuristics and hybrid
metaheuristics for the travel salesman problem is presented. Four
single classical metaheuristics (genetic algorithm, memetic
algorithm, iterated local search, and simulated annealing) were
used. In addition, hybrid variations using nine different heuristic
techniques for the local search, the mutation, and the
intensification were used. The performance analysis was made
using the Friedman test, and for the simulated annealing and local
search algorithms statistical evidence was found that hybridization
provides a difference in performance, while no evidence was found
for the genetic and memetic algorithms. Up to six combinations
were found to improve performance, five of them based on local
search and one more based on simulated annealing.
Keywords— Metaheuristics; Heuristics; Traveling Salesman
Problem; combinatorial optimization; Friedman.
I. INTRODUCTION
Optimization problems involving many finite solutions are
of interest in a wide range of fields. This kind of problems are
classified as combinatorial optimization (CO), and to find the
globally optimal solution it is theoretically possible to
enumerate and evaluate each one of the solutions. But this
approach becomes intractable rapidly due to the exponential
growth of most solution spaces. Metaheuristics allow us to
simplify this job since they can find solutions close the
optimal in a reasonable time. Metaheuristics evolved because
most modern problems are computationally intractable,
needing heuristic guidance to find good solutions, but not
necessarily the most optimal. Some of the must use techniques
include genetic algorithms (GA), memetic algorithm (MA),
iterated local search (ILS), and simulated annealing (SA).
The travel salesman problem is a CO problem that has been
studied extensively, and it is often used as a test for new
optimization algorithms. Heuristic techniques have been tested
in different instances of the travel agent algorithm. In this work
a Friedman's test analysis was performed to probe if the use of
various methods like local search, mutation, and intensification
have an impact on performance as compared to the single
classical heuristics, the results are ranked according to their
performance.
II. THEORICAL DESCRIPTION
A. Heuristics
A heuristic technique is a process that, for a particular
problem, offers a good solution, despite the fact the solution
might not be optimal. Generally speaking, these techniques are
applied to problems that are difficult to solve, and where it is
important to find a quick and easy solution (Zanakis & Evans,
1981).
The heuristics used in this work are: Random Insertion
(Twors), Reverse Sequence Mutation (RSM), Thors (Abdoun,
Abouchabaka, & Tajani, 2012), OPT (Blazinskas &
Misevicius, 2009; Gutin & Punnen, 2007), georeferenced
intersection, Centre inverse mutation, Closest insertion and
Throas Mutation (Abdoun et al., 2012). These techniques are
intended to help in the solution of the travel agent
problem.(Bang-Jensen, Gutin, & Yeo, 2004), (Bendall &
Margot, 2006), (Talbi, 2009)(Dorigo & Tutzle, 2010), (Cook,
2011).
B. Metaheuristics
Metaheuristics are an iterative search strategy that guides
the process over the search space in the hope of finding the
optimal solution (Glover,1986). In general, metaheuristics try
to combine basic heuristic methods in higher level frameworks
aimed at efficiently and effectively exploring a search space.
Metaheuristics allow working at large scales by obtaining a
satisfactory solution in a reasonable time. When working with
metaheuristics there is no warranty that a global optimal will be
found, not even a solution close to the extremes. However,
these techniques have gained popularity in the last 20 years
since in several applications they have shown efficiency and
efficacy in the solution of large complex problems. In
metaheuristic design, two opposing criteria must be met: the
first one, an efficient exploration of the search space, or
diversification; and the second one focusing on a local region
where a good solution has been found, or intensification.
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 6, June 2018
195 https://sites.google.com/site/ijcsis/
ISSN 1947-5500

2. C. Genetic Algorithms
The GAs are adaptive exploration methods that can be used
in the search for a solution and optimization. The GAs are
based on the natural selection that drives the dynamics of
biological populations. GAs use a probabilistic selection of
individuals for the crossover operation. The replacement of the
best individuals is generational, which means that the children
systematically replace the parents. The crossover operation is
based on n-points or steady state, while mutation is performed
as the interchange of bits or characteristics.
D. Memetic Algorithms
An MA is composed of two parts: The genetic algorithm
and the local search. The local search is a modification of an
individual or the total population by copying and perturbing
the individual to obtain an individual with a better fitness.
E. Local Iterated Search
The ILS uses an embedded local-search component
iteratively restarting it from different promising areas in the
search-space. The solutions obtained are better than using
random runs without heuristics.
F. Simulated Annealing
This local search algorithm ILS uses an embedded local-
search component iteratively restarting it from different
promising areas in the search-space. The solutions obtained
are better than using random runs without heuristics.
G. Travel Salesman Problem
The travel salesman is a NP-hard problem, and it is one of
the best-known combinatorial optimization problems. Given n
cities and the geographical distances between each one of
them, the task is to find the shortest closed tour in which each
city is visited exactly once.
H. Friedman’s Test
The Friedman’s test is a multiple comparison test in which
the null hypothesis is that the performance of all the
algorithms under comparison is similar. It yields a ranking of
the algorithms according to their performance with respect to
the control algorithm. To corroborate the ranking a post-hoc
procedure is required to identify the differences between the
control algorithm and the others.
I. Holm’s procedure
The Holm’s procedure is used for post-hoc testing, this
method is designed for multiple hypothesis testing iteratively
accepting or rejecting each one. The procedure begins by
ordering the m hypothesis by respective p-value, then each one
of the p-values is compared to their alpha values as calculated
from:
)1( +−
=
im
i

 (1)
where i is the index of the ordered values of p. If the p-value is
smaller the hypothesis is rejected, and the rest of the p-values
are compared using the current alpha value as threshold. This
procedure rejects from H1 to H(i-1) until pi > αi
Fig. 1. Multiple EAs and SAs running in parallel.
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 6, June 2018
196 https://sites.google.com/site/ijcsis/
ISSN 1947-5500

3. III. NUMERICAL EXPERIMENTS
Numerical experiments were performed under equal
conditions, the results are presented in Table 1. A total of 99
combinations of Metaheuriscs - Heuristcs were analyzed: 9
GAs for mutation, 9 MA for mutation and intensification, 9
SA for local search and a combination of this for a total of 72,
were used TSPLIB and are: KROA100, KROB100,
KROC100, KROD100 and KROE100.
))(log*322.3(1 10 nC += (2)
IV. RESULTS
Using the Friedman test the medians were compared with
α = 0.01, n=5, k=99, and the following hypothesis:
•H0 = the algorithms offer similar results
•Ha = the combinations offer different results.
Table 2 shows the results for the test on whether the use of
different heuristics has an impact on performance. The best
result has a significant performance improvement with respect
to the others and is close to the known optimal value.
To evaluate this, Friedman test was used considering α =
0.01, n=5, k=99, and the following hypothesis:
•H0 = the algorithms offer similar results
•Ha = the combinations offer different results.
Obtaining a p-value of 6.80e-53, thus rejecting Ho. By
using this test, the best combination was SA with RSM
heuristic as local search. To corroborate the 10% of the
minima values are taken, the results are shown in Table 3.
Using Friedman test using α = 0.01, n=5, k=9 and the
hypothesis:
•H0 = the algorithms offer similar results
•Ha = the combinations offer different results.
The result is a p-value of 4.41e-06, thus rejecting Ho.
Corroborating that the best combination was SA with RSM
heuristic as local search.
a..
p-value < α
b.
El p-value > α
c.
Combinations with performance close to the control
algorithm.
Table 1. Parameters used for numerical experiments
Parameter Metaheuristics
GA MA LIS SA
Dimension 100 100 100 100
Population 526 a
526 a
1 1
Stop criteria 100,000 b
100,000 b
100,000 b
100,000 b
Experiments 33 33 33 33
Selection Vasconcelos Vasconcelos - -
Crossover k-opt c
k-opt c
- -
Mutation 1% 1%
Elitism 5% 5% - -
Intensification - 10 iterations - -
Degrees - - - 36
Mk - - - 20
a.
Based on eq. (2), b.
Function calls, c.
k=1
Table 2. Friedman’s test results
Parameter Metaheuristics
GA MA LIS SA
P-valor 2.18E-02 4.76E-01 5.11E-39 3.61E-06
k 9 9 72 9
Best
combination
- a
- a
RSM-
Centre
Inverse b
RSM b
a
Since p-value > α, ∴ There is no evidence to reject H0. Therefore, a better combination
is not determined.
b
Since p-valor < α, ∴ It is rejected H0. Therefore, the test indicates the combination that
you consider best.
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 6, June 2018
197 https://sites.google.com/site/ijcsis/
ISSN 1947-5500

4. V. CONCLUSIONS
Table 4. Holm procedure with RSM
Combinations Values
Statistics z P-value α adjusted H0 rejected?
Opt3_RSM 4.9057789 9.303E-07 0.00125 yes a
Opt2_RSM 4.1311822 3.609E-05 0.0014285 yes a
RSM_Opt3 3.7438839 0.0001811 0.0016666 yes a
Throas_RSM c
2.9692872 0.0029849 0.002 no b
RSM_Throas c
2.1946905 0.0281858 0.0025 no b
RSM_Opt2 c
1.8073922 0.0707011 0.0033333 no b
RSM_ClosestInsertion c
1.0327955 0.3016995 0.005 no b
RSM_CentreInverse c
0.1290994 0.8972789 0.01 no b
a. p-value < α b. El p-value > α
c. Combinations with performance close to the control algorithm
Fig. 2. Graph of the results.
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 6, June 2018
198 https://sites.google.com/site/ijcsis/
ISSN 1947-5500

5. V. CONCLUSIONS
Based on the results obtained, for the GA and MA there is
no statistical evidence that applying heuristics for the mutation
operator would affect the performance.
On the other hand, according to the Friedman’s test, ILS
and SA are improved using different heuristics in the local
search or in the perturbation.
The SA with RSM in the local search ranked as the best
algorithm for the combinations under study.
After the post-hoc procedure it is concluded that there no
statistical evidence that 5 combinations produce an effect on
the final performance as compared to the control algorithm.
The best combinations are presented in Table 4.
As future work it will be found the adequate stop criteria
since it is known that this parameter might affect the
performance of population-based algorithms.
ACKNOWLEDGMENT
Authors whould like to acknowledge CONACYT, TecNM and
Instituto Tecnológico de León for the given support.
REFERENCES
Abdoun, O., Abouchabaka, J., & Tajani, C. (2012). Analyzing the
Performance of Mutation Operators to Solve the Travelling Salesman
Problem. International Journal of Emerging Sciences, 2(1), 61–77.
Bang-Jensen, J., Gutin, G., & Yeo, A. (2004). When the greedy algorithm
fails. Discrete Optimization, 1(2), 121–127.
http://doi.org/10.1016/j.disopt.2004.03.007
Bendall, G., & Margot, F. (2006). Greedy-type resistance of combinatorial
problems. Discrete Optimization, 3(4), 288–298.
http://doi.org/10.1016/j.disopt.2006.03.001
Blazinskas, A., & Misevicius, A. (2009). Combining 2-Opt, 3-Opt and 4-Opt
With K-Swap-Kick Perturbations for the Traveling Salesman Problem.
Kaunas University of Technology, Department of Multimedia
Engineering, Studentu St, 50–401.
Cook, W. (2011). In pursuit of the traveling salesman: mathematics at the
limits of computation. Chemistry & …, 635–638. Retrieved from
http://books.google.com/books?hl=en&lr=&id=S3bxbr_-
qhYC&oi=fnd&pg=PP2&dq=In+pursuit+of+the+traveling+salesman+
:mathematics+at+the+limits+of+computation&ots=jDzfyEkzKs&sig=
13_GiNHiPpLZDgM4p6tAtI4OOLs
Dorigo, M., & Tutzle, T. (2010). Handbook of Metaheuristics. In
M.Gendreau, J.-Y. Potvin (eds.), Handbook of Metaheuristics,
International Series in Operations Research and Management Science
(Vol. 146, p. 648). http://doi.org/10.1007/978-1-4419-1665-5.
Gutin, G., & Punnen, A. (2007). The Traveling Salesman Problem and its
Variations. … of Combinatorial Optimization: Problems and …, 749.
http://doi.org/10.1007/b101971
Talbi, E. G. (2009). Metaheuristics: From Design to Implementation.
Metaheuristics: From Design to Implementation.
http://doi.org/10.1002/9780470496916
Zanakis, S. H., & Evans, J. R. (1981). Heuristic “Optimization”: Why, When,
and How to Use It. Interfaces, 11(5), 84–91.
http://doi.org/10.1287/inte.11.5.84o. 2, pp. 415–420, 2008.
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 6, June 2018
199 https://sites.google.com/site/ijcsis/
ISSN 1947-5500