1. Chapter 26
All-Pairs Shortest Paths
Dr. Muhammad Hanif Durad
Department of Computer and Information Sciences
Pakistan Institute Engineering and Applied Sciences
hanif@pieas.edu.pk
Some slides have bee adapted with thanks from some other lectures
available on Internet. It made my life easier, thanks to Allah
Almighty .
2. Lecture Outline (1/2)
All-Pairs Shortest Path
Extending SSP
The Floyd-Warshall algorithm
The structure of a shortest path
A recursive solution
Computing weights
Constructing Shortest Path
Analysis
Transitive closure of a directed graph
22. All Pairs Shortest Path.ppt, P-8
3. Lecture Outline (2/2)
Johnson’s algorithm for sparse graphs
Idea
Preserving Shortest Paths by Reweighting
Producing nonnegative weights by
reweighting
Algorithm
Complexity
4. All-Pairs Shortest Path Problem
Generalization of the single source shortest-path problem.
Simple solution: run the shortest path algorithm for each
vertex complexity is
O(|E|.|V|.|V|) = O(|V|4) for Bellman-Ford
O(|E|.lg |V|.|V|) = O(|V|3 lg |V|) for Dijsktra.
Can we do better? Intuitively it would seem so, since
there is a lot of repeated work exploit the optimal sub-
path property.
We indeed can do better: O(|V|3).
Dr. Hanif Durad 4
lect16.ppt, P-2/39
See next slide for
details
5. All pair shortest Path Problem
Dr. Hanif Durad 5
The easiest way!
Iterate Dijkstra’s and Bellman-Ford |V| times!
Dijkstra:
O(VlgV + E) --> O(V2lgV + VE)
Bellman-Ford:
O(VE) ------> O(V2E)
Faster-All-Pairs-Shortest-Paths:
O(V3lgV)
O(V3)
O(V4)
On dense
graph
22. All Pairs Shortest Path.ppt
Edges (?)=V2
9. All shortest-paths: representation
We will use matrices to represent the graph, the
shortest path lengths, and the predecessor sub-graphs.
Edge matrix: entry (i,j) in adjacency matrix W is the
weight of the edge between vertex i and vertex j.
Shortest-path lengths matrix: entry (i,j) in L is the
shortest path length between vertex i and vertex j.
Predecessor matrix: entry (i,j) in Π is the predecessor
of j on some shortest path from i (null when i = j or
when there is no path).
Dr. Hanif Durad 9
11. Printing Shortest Path from
vertex i to vertex j
Dr. Hanif Durad 11
PRINT-ALL-PAIRS-SHORTEST-PATH(, ,i j)
1 if i j
2 then print i
3 else if ij NIL
4 then print “no path from” i to j “exists”
5 else PRINT-ALL-SHORTEST-PATH( , , )i ij
6 Print j
15. The structure of a shortest path
1. All sub-paths of a shortest path are shortest paths.
Let p = <v1, .. vk> be the shortest path from v1 to vk.
The sub-path between vi and vj, where 1 ≤ i,j ≤ k, pij
= <vi, .. vj> is a shortest path.
2. The shortest path from vertex i to vertex j with at
most m edges is either:
the shortest path with at most (m-1) edges (no
improvement)
the shortest path consisting of a shortest path within the (m-
1) vertices + the weight of the edge from a vertex within the
(m-1) vertices to an extra vertex m.
16. 16
Recursive solution to all-shortest
paths
}){min,min( )1(
1
)1()(
kj
m
ik
nk
m
ij
m
ij wlll
}{min )1(
1
kj
m
ik
nk
wl
ji
ji
lij
0)0(
Let l(m)
ij be the minimum weight of any path from
vertex i to vertex j that has at most m edges.
When m=0:
For m ≥ 1, l(m)
ij is the minimum of l(m–1)
ij and the
shortest path going through the vertices neighbors:
17. 17
All-shortest-paths: solution
Let W=(wij) be the edge weight matrix and
L=(lij) the all-shortest shortest path matrix
computed so far, both n×n.
Compute a series of matrices L(1), L(2), …,
L(n–1) where for m = 1,…,n–1, L(m) = (l(m)
ij) is
the matrix with the all-shortest-path lengths
with at most m edges. Initially, L(1) = W, and
L(n–1) containts the actual shortest-paths.
Basic step: compute L(m) from L(m–1) and W.
18. 18
Algorithm for extending all-shortest
paths by one edge: from L(m-1) to L(m)
EXTEND-SHORTEST-PATHS(L=(lij),W)
n rows[L]
Let L’ =(l’ij) be an n×n matrix.
for i 1 to n do
for j 1 to n do
l’ij ∞
for k 1 to n do
l’ij min(l’ij, lik + wkj)
return L’
Complexity: Θ(|V|3)
19. 19
This is exactly as matrix
multiplication!
Matrix-Multiply(A,B)
n rows[A]
Let C =(cij) be an n×n matrix.
for i 1 to n do
for j 1 to n do
cij 0 (l’ij ∞)
for k 1 to n do
cij cij+ aik.bkj (l’ij min(l’ij, lij + wkj))
return L’
min
)(
)1(
cl
bw
al
m
m
25. 25
Improved All-shortest Paths
Algorithm
The goal is to compute the final L(n–1), not all the L(m)
We can avoid computing most L(m) as follows:
L(1) = W
L(2) = W.W
L(4) = W4 = W2.W2
… )12()12()2()2( )1lg()1lg()1lg()1lg(
.
nnnn
WWWL
Since the final product is equal to L(n–1)
12 )1lg(
nn
only |lg(n–1)| iterations are necessary!
repeated squaring
28. Assignment 1
Problems 25.1-3, 25.1-8 25.1-10 Page
627628
C++ coding of
EXTEND-SHORTEST-PATHS,
SLOW-ALL-PAIRS-SHORTEST-PATHS and
FASTER-ALL-PAIRS-SHORTEST-PATHS
Last Date
Thursday -29 June 2010
Dr. Hanif Durad 28
29. All pair shortest Path Problem
Dr. Hanif Durad 29
The easiest way!
Iterate Dijkstra’s and Bellman-Ford |V| times!
Dijkstra:
O(VlgV + E) --> O(V2lgV + VE)
Bellman-Ford:
O(VE) ------> O(V2E)
Faster-All-Pairs-Shortest-Paths:
O(V3lgV)
O(V3)
O(V4)
On dense
graph
22. All Pairs Shortest Path.ppt
31. 31
Floyd-Warshall algorithm
Assumes there are no negative-weight cycles.
Uses a different characterization of the
structure of the shortest path. It exploits the
properties of the intermediate vertices of the
shortest path.
Runs in O(|V|3).
32. 32
Structure of the shortest path (1)
An intermediate vertex vi of a simple path p=<v1,..,vk> is
any vertex other than v1 or vk.
Let V={1,2,…,n} and let K={1,2,…,k} be a subset for k
≤ n. For any pair of vertices i,j in V, consider all paths
from i to j whose intermediate vertices are drawn from
K. Let p be the minimum-weight path among them.
i
k1
j
k2
p
33. 33
Structure of the shortest path (2)
1. k is not an intermediate vertex of path p:
All vertices of path p are in the set {1,2,…,k–1}
a shortest path from i to j with all intermediate
vertices in {1,2,…,k–1} is also a shortest path with all
intermediate vertices in {1,2,…,k}.
2. k is an intermediate vertex of path p:
Break p into two pieces: p1 from i to k and p2 from k to
j. Path p1 is a shortest path from i to k and path p2 is a
shortest path from k to j with all intermediate vertices
in {1,2,…,k–1}.
34. 34
Structure of the shortest path (3)
i
k j
p2
all intermediate vertices in
{1,2,…,k–1}
all intermediate vertices in
{1,2,…,k}
p1
all intermediate vertices in
{1,2,…,k–1}
45. Extracting Shortest Paths from π
Use PRINT-ALL-PAIRS-SHORTEST-PATH
Algorithm described earlier
Dr. Hanif Durad 45
46. 46
Transitive Closure (1/3)
Given a directed graph G=(V,E) with vertices V =
{1,2,…,n} determine for every pair of vertices (i,j) if
there is a path between them.
The transitive closure graph of G, G*=(V,E*) is such
that E* = {(i,j): if there is a path i and j}.
Represent E* as a binary matrix and perform logical
binary operations AND (/) and OR (/) instead of min
and + in the Floyd-Warshall algorithm.
47. 47
Transitive Closure (2/3)
( )
( 1) ( 1) ( 1)
0 if 0 and ( , )
1 if 0 and ( , )
( ) for 0
k
ij
k k k
ij ik kj
k i j E
t k i j E
t t t k
The definition of the transitive closure is:
The matrices T(k) indicate if there is a path with
at most k edges between s and i.
48. 48
Transitive Closure Algorithm (3/3)
Complexity: Θ(|V|3)
TRANSITIVE-CLOSURE(G)
1 | [ ]|n V G
2 for i 1 to n
3 do for j 1 to n
4 do if i j or ( , ) ( )i j E G
5 then tij
( )0
1
6 else tij
( )0
0
7 for k 1 to n
8 do for i 1 to n
9 do for j 1 to n
10 do t t t tij
k
ij
k
ik
k
kj
k( ) ( ) ( ) ( )
( ) 1 1 1
11 return T n( )
52. 25.3 Johnson’s algorithm for
sparse graphs
Makes clever use of Bellman-Ford and Dijkstra to do
All-Pairs-Shortest-Paths efficiently on sparse graphs.
Motivation
By running Dijkstra |V| times, we could do APSP in time
O(V2lgV +VElgV) (Modified Dijkstra), or
O (V2lgV +VE) (Fibonacci Dijkstra).
This beats O (V3) (Floyd-Warshall) when the graph is
sparse.
Problem: negative edge weights.
Dr. Hanif Durad 52
53. The Basic Idea
Reweight the edges so that:
1. No edge weight is negative.
2. Shortest paths are preserved. (A shortest path in the
original graph is still one in the new, reweighted
graph.)
An obvious attempt: subtract the minimum weight from
all the edge weights. E.g. if the minimum weight is -2:
-2 - -2 = 0
3 - -2 = 5 etc.
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 53
54. Counter Example
Subtracting the minimum weight from every
weight doesn’t work.
Consider:
Paths with more edges are unfairly penalized.
-2 -1
-2
0 1
0
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 54
55. Johnson’s Insight
Add a vertex s to the original graph G, with
edges of weight 0 to each vertex in G:
Assign new weights ŵ to each edge as follows:
ŵ(u, v) = w(u, v) + d(s, u) - d(s, v)
s 0
0
0
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 55
56. Question 1
Are all the ŵ’s non-negative? Yes:
Otherwise, s u v would be
shorter than the shortest path
from s to v.
s
u
v
w(u, v)
0),(),(),(
),(),(),(
vsusvuw
vsvuwus
dd
dd
:Rewriting
),(ˆ vuw
),(),(),( vsvuwus dd bemust
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 56
57. Question 2
Does the reweighting preserve shortest paths? Yes
Consider any path
kvvvp ,,, 21
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 57
58. Lemma 25.1 (Reweighting doesn’t change shortest paths)
Given a weighted directed graph G = (V, E) with weight function
w E R: , let h V R: be any function mapping vertices to real
numbers. For each edge ( , )u v E , define
( , ) ( , ) ( ) ( )w u v w u v h u h v .
Let P v v vk 0 1, ,..., be a path from vertex v0 to vk . Then
w P v vk( ) ( , ) d 0 if and only if ( ) ( , )w P v vk d 0 .
Also, G has a negative-weight cycle using weight function w iff G has
a negative weight cycle using weight function w.
Preserving Shortest Paths by
Reweighting (1/4)
25_All-Pairs Shortest Paths.ppt
Dr. Hanif Durad 58
59. Proof.
( ) ( ) ( ) ( )w P w P h v h vk 0
( ) ( , )
( ( , ) ( ) ( ))
( , ) ( ) ( )
( ) ( ) ( )
w P w v v
w v v h v h v
w v v h v h v
w P h v h v
i i
i
k
i i
i
k
i i
i i
i
k
k
k
1
1
1
1
1
1
1
0
0
Preserving Shortest Paths by
Reweighting (2/4)
25_All-Pairs Shortest Paths.ppt
Dr. Hanif Durad 59
60. w P v vk( ) ( , ) d 0 implies ( ) ( , )w P v vk d 0 .
Suppose there is a shorter path P' from v0 to vk using the weight
function w. Then ˆ ˆ( ') ( )w P w P .
Then
w P h v h v w P
w P w P h v h v
w P w P
k
k
( ' ) ( ) ( ) ( ' )
( ) ( ) ( ) ( )
( ' ) ( ).
0
0
We get a contradiction!
Preserving Shortest Paths by
Reweighting (3/4)
25_All-Pairs Shortest Paths.ppt
Dr. Hanif Durad 60
61. G has a negative-weight cycle using w iff G has a
negative-weight cycle using w.
Consider any cycle 0 1, ,..., kc v v v with v vk0 . Then
0
ˆ( ) ( ) ( ) ( ) ( )kw c w c h v h v w c .
and thus c has negative weight using w if and only if it has
negative weight using
Preserving shortest paths by
reweighting (4/4)
25_All-Pairs Shortest Paths.ppt
Dr. Hanif Durad 61
62. Question 3
How do we compute the d(s, v)’s?
Use Bellman-Ford.
This also tells us if we have a negative-weight
cycle.
8a-ShortestPathsMore.ppt
Dr. Hanif Durad 62
66. Johnson’s algorithm example
0
-1
0 -5
-4 0
4
10
0
2
13
0
2
0
s
5
1
0
4
0
Implementation of
the Bellman-Ford
algorithm, by
starting from s the
shortest distance
of each point.
2
1 3
4
5
Fig. 25.6 (b)
Dr. Hanif Durad 66
73. Johnson’s Algorithm Complexity
1. Find a vertex labeling h such that ŵ(u, v) ≥ 0 for all
(u, v) E by using Bellman-Ford to solve the
difference constraints
h(v) – h(u) ≤ w(u, v),
or determine that a negative-weight cycle exists.
• Time = O(VE).
2. Run Dijkstra’s algorithm from each vertex using ŵ.
• Time = O(VE + V2 lg V).
3. Reweight each shortest-path length ŵ(p) to produce
the shortest-path lengths w(p) of the original graph.
• Time = O(V2).
Total time = O(VE + V2 lg V).
lecture19.ppt
Dr. Hanif Durad 73
74. Assignment 3
Section Problem 25.3-6
Chapter Problems 25.1
C++ coding of
Johnson’s Algorithm
Last Date
Thursday -29 June 2010
Dr. Hanif Durad 74
75. 75
Other Graph Algorithms (1/3)
Many more interesting problems, including
network flow, graph isomorphism, coloring,
partition, etc.
Problems can be classified by the type of solution.
Easy problems: polynomial-time solutions O(f (n))
where f (n) is a polynomial function of degree at
most k.
Hard problems: exponential-time solutions O(f (n))
where f (n) is an exponential function, usually 2n.
lect16.ppt,
76. 76
Easy Graph Problems (2/3)
Network flow – maximum flow problem
Maximum bipartite matching
Planarity testing and plane embedding.
77. 77
Hard Graph Problems (3/3)
Graph and sub-graph isomorphism.
Largest clique, Independent set
Vertex Tour (Traveling Salesman problem)
Graph partition
Vertex coloring
However, not all is lost!
Good heuristics that perform well in most cases
Polynomial-time Approximation algorithms