COMBINATORIAL 
OPTIMIZATION 
Guided By: Prepared By 
Dhaval Jha Anjali Bidua 
Asst Professor 12bce155
Overview 
What is Optimization? 
What is Combinatorial Optimization? 
Applications of Combinatorial Optimization 
Identification of different Combinatorial Problems 
Minimum Spanning Tree 
Overview Of Travelling Salesman Problem
Optimization 
Optimization is technique of finding the optimal solution when 
more than one solutions are available. 
It is a live field of Mathematics. 
It can be better known by Integer Programming. 
Linear programming is a mathematical programming technique 
to optimize performance under a set of resource constraints.
There are various methods for solving optimization problems by 
integer programming. 
Some of them are: 
Graphical Method 
Simplex Method
Combinatorial Optimization 
Combinatorial optimization deals with algorithmic 
approaches to finding specified configurations or 
objects in finite structures such as directed and 
undirected graphs, hyper graphs, networks, matroids, 
partially ordered sets, and so forth.
Major Combinatorial Optimization Problems 
Minimum spanning tree 
Travelling salesman problem 
Vehicle routing problem 
Weapon target assignment problem 
Knapsack problem
Minimum Spanning Tree 
Given a connected, undirected graph, a spanning tree 
of that graph is a subgraph that is a tree and connects 
all the vertices together. 
A minimum spanning tree (MST) or minimum weight 
spanning tree is then a spanning tree with weight less 
than or equal to the weight of every other spanning 
tree.
Travelling Salesman Problem 
Given a set of cities and the distance between each 
possible pair, the Travelling Salesman Problem is to 
find the best possible way of ‘visiting all the cities 
exactly once and returning to the starting point’.
Vehicle Routing Problem 
The vehicle routing problem (VRP) is a combinatorial optimization and 
integer programming problem seeking to service a number of customers 
with a fleet of vehicles. 
VRP is an important problem in the fields of transportation, distribution and 
logistics. Often the context is that of delivering goods located at a central 
depot to customers who have placed orders for such goods. 
Implicit is the goal of minimizing the cost of distributing the goods. Many 
methods have been developed for searching for good solutions to the 
problem, but for all but the smallest problems, finding global minimum for 
the cost function is computationally complex.
Weapon Target Assignment problem 
It consists of finding an optimal assignment of a set of 
weapons of various types to a set of targets in order 
to maximize the total expected damage done to the 
opponent
Knapsack problem 
The problem often arises in resource allocation 
where there are financial constraints and is 
studied in fields such as combinatorics, 
computer science, complexity theory, 
cryptography and applied mathematics.
Minimum Spanning Tree 
Definition: A tree is a connected undirected graph with no 
simple circuits. 
A spanning tree of a graph is just a subgraph that contains all 
the vertices and is a tree. 
Spanning Tree properties: On a connected graph G=(V, E), a 
spanning tree: 
• is a connected subgraph 
• acyclic 
• is a tree (|E| = |V| - 1) 
• contains all vertices of G (spanning)
Minimum weight spanning tree (MST) 
On a weighted graph, A MST: 
connects all vertices through edges with least weights. 
has w(T) ≤ w(T’) for every other spanning tree T’ in G 
Adding one edge not in MST will create a cycle
Finding Minimum Spanning Tree 
Algorithms : 
Kruskal’s 
Prim’s
Compare Prim and Kruskal 
Complexity 
Kruskal: O(NlogN) 
comparison sort for edges 
Prim: O(NlogN) 
search the least weight edge for 
every vertice
MST Conclusion 
Kruskal’s has better running times if the number 
of edges is low, while Prim’s has a better running 
time if both the number of edges and the 
number of nodes are low. 
So, of course, the best algorithm depends on the 
graph and if you want to bear the cost of 
complex data structures.
Applications of MST 
Design of networks, including computer networks, 
telecommunications networks, transportation 
networks, water supply networks, and electrical grids 
Applications of MST in Sensor Networks 
A tree is formed from all sensor nodes to the sink so 
that the transmission power of all sensor nodes is 
minimum
Travelling Salesman Problem 
In the theory of computational complexity, the 
decision version of the TSP where, given a length L, 
the task is to decide whether the graph has any tour 
shorter than L.
TSP : An NP-HARD Problem! 
TSP is an NP-hard problem in combinatorial optimization 
studied in theoretical computer science. 
In many applications, additional constraints such as limited 
resources or time windows make the problem 
considerably harder. 
Removing the constraint of visiting each city exactly one 
time also doesn't reduce complexity of the problem.
Applications of TSP 
Planning 
Logistics 
Manufacture of microchips 
Slightly modified, it appears as a sub-problem in many areas, such as DNA sequencing 
Integrated circuit 
Mechanical arm
Solutions 
Exact solutions 
Heuristic algorithms 
Finding subproblems for the problem
Exact Solution 
Consider city 1 as the starting and ending point. 
Generate all (n-1)! Permutations of cities. 
Calculate cost of every permutation and keep track of 
minimum cost permutation. 
Return the permutation with minimum cost. 
Time Complexity: O(n!)
Heuristic Solution 
The greedy algorithm can be used to give a good 
but not optimal solution in a reasonably short 
amount of time. 
The greedy algorithm heuristic says to pick 
whatever is currently the best next step 
regardless of whether that precludes good steps 
later.
Dynamic Programming 
Division of Problem into Sub-Problems 
Memoization
An Example
An Example 
Let the given set of vertices be {1, 2, 3, 4,….n}. 
Let us consider 1 as starting and ending point of 
output. 
For every other vertex i (other than 1), we find 
the minimum cost path with 1 as the starting 
point, i as the ending point and all vertices 
appearing exactly once.
Let the cost of this path be cost(i), the cost of corresponding 
Cycle would be cost(i) + dist(i, 1) where dist(i, 1) is the distance 
from i to 1. Finally, we return the minimum of all [cost(i) + dist(i, 
1)] values. 
Let us define a term C(S, i) be the cost of the minimum cost 
path visiting each vertex in set S exactly once, starting at 1 and 
ending at i. 
We start with all subsets of size 2 and calculate C(S, i) for all 
subsets where S is the subset, then we calculate C(S, i) for all 
subsets S of size 3 and so on. Note that 1 must be present in 
every subset.
If size of S is 2, then 
S must be {1, i}, C(S, i) = dist(1, i) 
Else if size of S is greater than 2 
C(S, i) = min { C(S-{i}, j) + dis(j, i)} 
where j belongs to S, j != i and j != 1. 
For a set of size n, we consider n-2 subsets each of size n-1 such that all 
subsets don’t have nth in them. 
The total running time is O(n2*2n). The time complexity is much less 
than O(n!), but still exponential. Space required is also exponential. So 
this approach is also infeasible even for slightly higher number of 
vertices.
TSP Using Dynamic Programming
TSP Without Dynamic 
Programming
Sparse Graphs 
For 2 cities
With Dynamic Programming
Without Dynamic Programming
Dense Graphs 
For 6 cities
With Dynamic Programming
Without dynamic Programming
Conclusion 
• When we have a limited set of targets to 
meet, Tsp using Dynamic Programming is 
efficient. 
• But when we have a considerable more 
number of targets this concept is not much 
useful.
Various scenarios
Connected Graph
Complete Graph
References 
• Discrete Optimization". Elsevier. Jump up Jump up Cook, 
William. "Optimal TSP Tours". University of Waterloo. 
• Fredman, M. L.; Willard, D. E. (1994), "Trans-dichotomous 
algorithms for minimum spanning trees and shortest 
paths", Journal of Computer and System Sciences 
• Karger, David R.; Klein, Philip N.; Tarjan, Robert E. (1995), "A 
randomized linear-time algorithm to find minimum 
spanning trees", Journal of the Association for Computing 
Machinery 
• Applegate, D. L.; Bixby, R. M.; Chvátal, V.; Cook, W. J. (2006), 
The Traveling Salesman Problem 
• Bellman, R. (1962), "Dynamic Programming Treatment of 
the Travelling Salesman Problem", J. Assoc. Comput. Mach.

Combinatorial Optimization

  • 1.
    COMBINATORIAL OPTIMIZATION GuidedBy: Prepared By Dhaval Jha Anjali Bidua Asst Professor 12bce155
  • 2.
    Overview What isOptimization? What is Combinatorial Optimization? Applications of Combinatorial Optimization Identification of different Combinatorial Problems Minimum Spanning Tree Overview Of Travelling Salesman Problem
  • 3.
    Optimization Optimization istechnique of finding the optimal solution when more than one solutions are available. It is a live field of Mathematics. It can be better known by Integer Programming. Linear programming is a mathematical programming technique to optimize performance under a set of resource constraints.
  • 4.
    There are variousmethods for solving optimization problems by integer programming. Some of them are: Graphical Method Simplex Method
  • 5.
    Combinatorial Optimization Combinatorialoptimization deals with algorithmic approaches to finding specified configurations or objects in finite structures such as directed and undirected graphs, hyper graphs, networks, matroids, partially ordered sets, and so forth.
  • 6.
    Major Combinatorial OptimizationProblems Minimum spanning tree Travelling salesman problem Vehicle routing problem Weapon target assignment problem Knapsack problem
  • 7.
    Minimum Spanning Tree Given a connected, undirected graph, a spanning tree of that graph is a subgraph that is a tree and connects all the vertices together. A minimum spanning tree (MST) or minimum weight spanning tree is then a spanning tree with weight less than or equal to the weight of every other spanning tree.
  • 8.
    Travelling Salesman Problem Given a set of cities and the distance between each possible pair, the Travelling Salesman Problem is to find the best possible way of ‘visiting all the cities exactly once and returning to the starting point’.
  • 9.
    Vehicle Routing Problem The vehicle routing problem (VRP) is a combinatorial optimization and integer programming problem seeking to service a number of customers with a fleet of vehicles. VRP is an important problem in the fields of transportation, distribution and logistics. Often the context is that of delivering goods located at a central depot to customers who have placed orders for such goods. Implicit is the goal of minimizing the cost of distributing the goods. Many methods have been developed for searching for good solutions to the problem, but for all but the smallest problems, finding global minimum for the cost function is computationally complex.
  • 10.
    Weapon Target Assignmentproblem It consists of finding an optimal assignment of a set of weapons of various types to a set of targets in order to maximize the total expected damage done to the opponent
  • 11.
    Knapsack problem Theproblem often arises in resource allocation where there are financial constraints and is studied in fields such as combinatorics, computer science, complexity theory, cryptography and applied mathematics.
  • 12.
    Minimum Spanning Tree Definition: A tree is a connected undirected graph with no simple circuits. A spanning tree of a graph is just a subgraph that contains all the vertices and is a tree. Spanning Tree properties: On a connected graph G=(V, E), a spanning tree: • is a connected subgraph • acyclic • is a tree (|E| = |V| - 1) • contains all vertices of G (spanning)
  • 13.
    Minimum weight spanningtree (MST) On a weighted graph, A MST: connects all vertices through edges with least weights. has w(T) ≤ w(T’) for every other spanning tree T’ in G Adding one edge not in MST will create a cycle
  • 14.
    Finding Minimum SpanningTree Algorithms : Kruskal’s Prim’s
  • 15.
    Compare Prim andKruskal Complexity Kruskal: O(NlogN) comparison sort for edges Prim: O(NlogN) search the least weight edge for every vertice
  • 16.
    MST Conclusion Kruskal’shas better running times if the number of edges is low, while Prim’s has a better running time if both the number of edges and the number of nodes are low. So, of course, the best algorithm depends on the graph and if you want to bear the cost of complex data structures.
  • 17.
    Applications of MST Design of networks, including computer networks, telecommunications networks, transportation networks, water supply networks, and electrical grids Applications of MST in Sensor Networks A tree is formed from all sensor nodes to the sink so that the transmission power of all sensor nodes is minimum
  • 18.
    Travelling Salesman Problem In the theory of computational complexity, the decision version of the TSP where, given a length L, the task is to decide whether the graph has any tour shorter than L.
  • 19.
    TSP : AnNP-HARD Problem! TSP is an NP-hard problem in combinatorial optimization studied in theoretical computer science. In many applications, additional constraints such as limited resources or time windows make the problem considerably harder. Removing the constraint of visiting each city exactly one time also doesn't reduce complexity of the problem.
  • 20.
    Applications of TSP Planning Logistics Manufacture of microchips Slightly modified, it appears as a sub-problem in many areas, such as DNA sequencing Integrated circuit Mechanical arm
  • 21.
    Solutions Exact solutions Heuristic algorithms Finding subproblems for the problem
  • 22.
    Exact Solution Considercity 1 as the starting and ending point. Generate all (n-1)! Permutations of cities. Calculate cost of every permutation and keep track of minimum cost permutation. Return the permutation with minimum cost. Time Complexity: O(n!)
  • 23.
    Heuristic Solution Thegreedy algorithm can be used to give a good but not optimal solution in a reasonably short amount of time. The greedy algorithm heuristic says to pick whatever is currently the best next step regardless of whether that precludes good steps later.
  • 24.
    Dynamic Programming Divisionof Problem into Sub-Problems Memoization
  • 25.
  • 26.
    An Example Letthe given set of vertices be {1, 2, 3, 4,….n}. Let us consider 1 as starting and ending point of output. For every other vertex i (other than 1), we find the minimum cost path with 1 as the starting point, i as the ending point and all vertices appearing exactly once.
  • 27.
    Let the costof this path be cost(i), the cost of corresponding Cycle would be cost(i) + dist(i, 1) where dist(i, 1) is the distance from i to 1. Finally, we return the minimum of all [cost(i) + dist(i, 1)] values. Let us define a term C(S, i) be the cost of the minimum cost path visiting each vertex in set S exactly once, starting at 1 and ending at i. We start with all subsets of size 2 and calculate C(S, i) for all subsets where S is the subset, then we calculate C(S, i) for all subsets S of size 3 and so on. Note that 1 must be present in every subset.
  • 28.
    If size ofS is 2, then S must be {1, i}, C(S, i) = dist(1, i) Else if size of S is greater than 2 C(S, i) = min { C(S-{i}, j) + dis(j, i)} where j belongs to S, j != i and j != 1. For a set of size n, we consider n-2 subsets each of size n-1 such that all subsets don’t have nth in them. The total running time is O(n2*2n). The time complexity is much less than O(n!), but still exponential. Space required is also exponential. So this approach is also infeasible even for slightly higher number of vertices.
  • 29.
    TSP Using DynamicProgramming
  • 31.
    TSP Without Dynamic Programming
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.
    Conclusion • Whenwe have a limited set of targets to meet, Tsp using Dynamic Programming is efficient. • But when we have a considerable more number of targets this concept is not much useful.
  • 40.
  • 41.
  • 42.
  • 43.
    References • DiscreteOptimization". Elsevier. Jump up Jump up Cook, William. "Optimal TSP Tours". University of Waterloo. • Fredman, M. L.; Willard, D. E. (1994), "Trans-dichotomous algorithms for minimum spanning trees and shortest paths", Journal of Computer and System Sciences • Karger, David R.; Klein, Philip N.; Tarjan, Robert E. (1995), "A randomized linear-time algorithm to find minimum spanning trees", Journal of the Association for Computing Machinery • Applegate, D. L.; Bixby, R. M.; Chvátal, V.; Cook, W. J. (2006), The Traveling Salesman Problem • Bellman, R. (1962), "Dynamic Programming Treatment of the Travelling Salesman Problem", J. Assoc. Comput. Mach.