2. Simulated Annealing
• It is a very general search technique.
• Try to avoid being trapped in local minimum by making
probabilistic moves.
• The method was first proposed by Metropolis (1953).
• Kirkpatrick et al. (1982) later improved the SA method
applied optimization problems
32. Basic Idea of Simulated Annealing
• Inspired by the Annealing Process
• The process of carefully cooling molten metals in order to
obtain a good crystal structure.
• First, metal is heated to a very high temperature, the
atoms have a higher energy state and a high possibility
to re-arrange the crystal structure.
• Then slowly cooled, the atoms have a lower and lower
energy state and a smaller and smaller possibility to re-
arrange the crystal structure
33. Simulated Annealing
• Analogy
• Metal Optimization Problem
• Energy State Cost Function
• Temperature Control Parameter
• A completely ordered crystal structure
the optimal solution for the problem
Global optimal solution can be achieved as long as the
cooling process is slow enough.
34. Metropolis Loop
• The essential characteristic of simulated annealing
• Determining how to randomly explore new solution,
reject or accept the new solution
at a constant temperature T.
• Finished until equilibrium is achieved.
35. Algorithm
Initialize initial solution x0 , highest temperature Th, and
coolest temperature Tl
T= Th
When the temperature is higher than Tl
While not in equilibrium
Search for the new solution x1
Accept or reject x1 according to Metropolis Criterion
End
Decrease the temperature T, according to cooling schedule
End
36. Metropolis Criterion
• Let
• x0 be the current solution and x1 be the new solution
• E(x0), E(x1) be the energy state (cost) of x0 and x1 .
• K=1 (Boltzmann constant)
• Probability Paccept = exp [-(E(x1)-E(x0))/ KT]
• Let r=Random (0,1)
• Unconditional accepted if
• E(x1) < E(x0), the new solution is better
• Probably accepted if
• E(x1) >= E(x0), the new solution is worse . Accepted only
when r < Paccept
37. Control Parameters
• Definition of equilibrium
• Cannot yield any significant improvement after certain
number of loops
• A constant number of loops
• Annealing schedule (i.e. How to reduce the
temperature)
• A constant value, T’ = T - Td
• A constant scale factor, T’= T * Rd
x A scale factor usually can achieve better performance