Upcoming SlideShare
×

# Algorithm Design and Complexity - Course 11

2,967 views

Published on

Published in: Education
3 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
2,967
On SlideShare
0
From Embeds
0
Number of Embeds
357
Actions
Shares
0
0
0
Likes
3
Embeds 0
No embeds

No notes for slide

### Algorithm Design and Complexity - Course 11

1. 1. Algorithm Design and Complexity Course 11
2. 2. Overview All-Pairs Shortest Paths (APSP) Using SSSP Algorithms Simple DP Algorithms Floyd-Warshall Algorithm Johnson’s Algorithm Transitive Closure of a Graph
3. 3. All-Pairs Shortest Paths (APSP) G(V, E) (un)directed, connected and weighted graph The weight (cost) function w: E → R w(u, v) = the weight of the edge (u, v) Adjacency matrix of weights W = [w(i, j)] ; 1 <= i, j <= n  w(i, j) = 0 if i = j  w(i, j) = INF if i != j and (i, j)E  w(i, j) = weight of the edge (i, j) if i != j and (i, j)E Compute the shortest paths between any two vertices in the graph Use a matrix D = [d(i, j)] ; 1 <= i, j <= n  We want d(i, j) = δ(i, j) = weight of the shortest path from i to j
4. 4. APSP – Predecessors We also compute a matrix of predecessors P = [p(i, j)] ; 1 <= i, j <= n  p(i, j) is the predecessor of j on the shortest path i..j  p(i, j) = NIL if there isn’t any path between i and j Therefore to find out the vertices on the shortest path between any two vertices, u and v, u..v:  1. Start from v’ = v  2. Go to v’ = p(u, v’)  3. If (v’ == u) then stop  4. Else go to step 2 (p(u,v), v), (p(u, p(u,v)), p(u,v))… (u, p(u, … p(u,v)))
5. 5. Solutions for APSP1. Use SSSP algorithms called n times  Considering each vertex as a source2. Use specialized algorithms  Try to compute the matrices D and P directly There is no algorithm that works best for all cases Consider the best choice given the problem needed to be solved  Dense vs. sparse graph, negative vs. positive weights  Special cases: DAGs  Etc.
6. 6. Using SSSP Algorithms for APSP For any type of graph Use Bellman-Ford – n times  n * (n*m) = (n2*m)  Dense graphs: (n4)  Sparse graphs: (n3)  We want to improve it! Using Dijkstra – n times  Only for positive weighted edges  Fibonacci heaps: n * (n*logn + m) = (n2*logn + n*m)  Dense graphs: (n3)  Sparse graphs: (n2*logn)  Otherwise, choose between binary heap and arrays for the best solution depending on the graph
7. 7. Specific APSP Algorithms The specific APSP algorithms, should work better than the previous solutions We want (n3) for any kind of graph  Floyd-Warshall algorithm Maybe find improvements for specific types of graphs  (n2*logn) for sparse graphs with negative weights  Johnson’s algorithm For some graphs, the SSSP solutions is the best one  E.g. for DAGS, the SSSP algorithm works great with minor improvements Use dynamic programming for designing these algorithms
8. 8. APSP DP Algorithms Use the property: any subpath of a shortest path is also a shortest path! What kind of sub-problems?1. Determine the shortest paths that contain at most k edges! (Simple DP algorithms)2. Determine the shortest paths that contain only the first k vertices as intermediate vertices on the SP (Floyd-Warshall algorithm) Start with k = 0 and then increase it!
9. 9. Simple DP algorithms Compute the APSP that contain at most k edges on the determined SP! Use L(k)[i, j] = the weight of the SP from vertex i to vertex j that contains <= k edges Start with k = 0 (stop condition for the recursion)  L(0)[i, j] = 0 if i = j  L(0)[i, j] = INF if i != j Use the following recursion for k >= 1 L(k)[i, j] = min(L(k-1)[i, j] , mink=1..n(L(k-1)[i, k] + w(k, j))) = mink=1..n(L(k-1)[i, k] + w(k, j)) because w(j, j) = 0
10. 10. Simple DP Algorithm for APSP Can also compute the predecessor matrix as well  Exercise: how to compute it! Verify the recursive formula when k = 1 L(1)[i, j] should be w(i, j) But L(1)[i, j] = mink=1..n(L(0)[i, k] + w(k, j)) = L(0)[i, i] + w(i, j) (the only non-INF value) = w(i, j)
11. 11. Simple DP Algorithm for APSP (2) There are at most n – 1 edges on each shortest path Therefore, we want to compute L(n-1) Afterwards, the matrix should not change anymore: L(n-1) = L(n) = L(n+1) = … δ(i, j) = L(n-1)[i, j] = L(n)[i, j] = L(n+1)[i, j]= … Start from L(1) = W Compute the solution in a bottom-up manner L(1), L(2), …, L(k), …, L(n-1)
12. 12. Simple DP Algorithm – PseudocodeSLOW-APSP(G, W) n = |V[G]| L[1] = W FOR (m=2; m < n; m++) L[m] = EXPAND(L[m-1], W, n) RETURN L[n-1]EXPAND (L, W, n) L’ = new matrix(n, n) FOR (i=1; i <= n; i++) FOR (j=1; j <= n; j++) L’[i][j] = INF FOR (k=1; k <= n; k++) L’[i][j] = min(L’[i][j], L[i][k] + w[k][j]) RETURN L’Time complexity: EXPAND - (n3) SLOW-APSP - (n4)  not very good! Same as Bellman-Ford - n times!Space complexity: uses n matrices - (n3)  can be reduced by using the same matrix L
13. 13. Improved Simple DP Algorithm Improve the way L(n-1) is computed Instead of computing: L(1), L(2), …, L(k), …, L(n-1) Why not compute? L(1), L(2), L(4), …, L(2^k), …, L(2^r) Stop when r = ceiling(log(n-1)) >= n-1, but this is ok! EXPAND is similar to matrix multiplication C = A * B  L A  W B  L’ C  min +  + *  INF 0
14. 14. Improved DP Algorithm – PseudocodeFAST-APSP(G, W) n = |V[G]| L[1] = W m=1 FOR (; m < n; m=2*m) L[2*m] = EXPAND(L[m], L[m], n) RETURN L[m]EXPAND (L, W, n) L’ = new matrix(n, n) FOR (i=1; i <= n; i++) FOR (j=1; j <= n; j++) L’[i][j] = INF FOR (k=1; k <= n; k++) L’[i][j] = min(L’[i][j], L[i][k] + w[k][j]) RETURN L’Time complexity: EXPAND - (n3) FAST-APSP - (n3 * logn)  Still not very good! But better than Bellman-Ford - n times!Space complexity: uses n matrices - (n3)  can be reduced by using the same matrix L
15. 15. Floyd-Warshall Algorithm Use another DP formulation Given a path p = <v1, v2, … , vj>  Any vertex except v1 and vj are intermediate vertices Sub-problem: which is the shortest path between any two vertices that contain intermediate vertices in the set {1, 2, … , k} ? D(k) = (D(k) [i, j]) for all 1 <= i, j <= n We want to compute D(n)
16. 16. Floyd-Warshall – Recursive Formulation Initialization (stop condition for the recursion): D(0) = W If no intermediate vertices are allowed, the best path between any two vertices is either the weight of the edge (if it exists) or INF Recursive formulation: D(k)[i, j] = min(D(k-1)[i, j], D(k-1)[i, k] + D(k-1)[k, j]) Choose between:  The shortest path between i and j that contains intermediate vertices in {1, 2, … , k-1}  The sum of the shortest paths from i to k and from k to j that contain intermediate vertices in {1, 2, … , k-1}
17. 17. Floyd-Warshall – Recursive Formulation The recursive formulation can be proved by induction  On whiteboard Can also compute P(k)  How? Compute the solution in a bottom-up fashion D(0), D(1),…, D(k),…, D(n)
18. 18. Floyd-Warshall – PseudocodeFLOYD-WARSHALL(G, W) n = |V[G]| D[0] = W FOR (i = 1; i <= n; i++) FOR (j = 1; j <= n; j++) IF (w(i, j) != INF) P[0][i][j] = i ELSE P[0][i][j] = NIL FOR (k = 1; k <= n; k++) FOR (i = 1; i <= n; i++) FOR (j = 1; j <= n; j++) IF (D[k-1][i][j] < D[k-1][i][k] + D[k-1][k][j]) D[k][i][j] = D[k-1][i][j] P[k][i][j] = P[k-1][i][j] ELSE D[k][i][j] = D[k-1][i][k] + D[k-1][k][j] P[k][i][j] = P[k-1][k][j] RETURN D[n]Time complexity: (n3)  good for dense graphs and for graphs with negative weightsSpace complexity: (n3)  can be reduced to (n2) by using a single D matrix and a single P matrix
19. 19. Example (1) 0 3 8 ∞ -4 2 3 4 ∞ 0 ∞ 1 7 7 8 D(0) = ∞ 4 0 ∞ ∞ 1 3 2 ∞ -5 0 ∞ -4 2 1 ∞ ∞ ∞ 6 0 -5 5 4 6 nil 1 1 nil 1 nil nil nil 2 2 P(0) = nil 3 nil nil nil 4 nil 4 nil nil nil nil nil 5 nil Proiectarea Algoritmilor 2010
20. 20. Example (2) 0 3 8 ∞ -4 0 3 8 ∞ -4 ∞ 0 ∞ 1 7 ∞ 0 ∞ 1 7D= ∞ 4 0 ∞ ∞ D= ∞ 4 0 ∞ ∞ 2 ∞ -5 0 ∞ 2 5 -5 0 -2 2 ∞ ∞ ∞ 6 0 3 4 ∞ ∞ ∞ 6 0 7 8 1 3 D(0), P(0) D(1), P(1) -4 1 2 -5 nil 1 1 nil 1 5 6 4 nil 1 1 nil 1 nil nil nil 2 2 nil nil nil 2 2p = nil 3 nil nil nil p = nil 3 nil nil nil 4 nil 4 nil nil 4 1 4 nil 1 nil nil nil 5 nil nil nil nil 5 nil Proiectarea Algoritmilor 2010
21. 21. Example (3) 0 3 8 ∞ -4 0 3 8 4 -4 ∞ 0 ∞ 1 7 ∞ 0 ∞ 1 7D= ∞ 4 0 ∞ ∞ D= ∞ 4 0 5 11 2 5 -5 0 -2 2 5 -5 0 -2 2 ∞ ∞ ∞ 6 0 3 4 ∞ ∞ ∞ 6 0 7 8 1 3 D(1), P(1) D(2), P(2) -4 1 2 -5 nil 1 1 nil 1 5 6 4 nil 1 1 2 1 nil nil nil 2 2 nil nil nil 2 2p = nil 3 nil nil nil p = nil 3 nil 2 2 4 1 4 nil 1 4 1 4 nil 1 nil nil nil 5 nil nil nil nil 5 nil Proiectarea Algoritmilor 2010
22. 22. Example (4) 0 3 8 4 -4 0 3 8 4 -4 ∞ 0 ∞ 1 7 ∞ 0 ∞ 1 7D= ∞ 4 0 5 11 D= ∞ 4 0 5 11 2 5 -5 0 -2 2 -1 -5 0 -2 2 ∞ ∞ ∞ 6 0 3 4 ∞ ∞ ∞ 6 0 7 8 1 3 D(2), P(2) D(3), P(3) -4 1 2 -5 nil 1 1 2 1 5 6 4 nil 1 1 2 1 nil nil nil 2 2 nil nil nil 2 2p = nil 3 nil 2 2 p = nil 3 nil 2 2 4 1 4 nil 1 4 3 4 nil 1 nil nil nil 5 nil nil nil nil 5 nil Proiectarea Algoritmilor 2010
23. 23. Example (5) 0 3 8 4 -4 0 3 -1 4 -4 ∞ 0 ∞ 1 7 3 0 -4 1 -1D= ∞ 4 0 5 11 D= 7 4 0 5 3 2 -1 -5 0 -2 2 -1 -5 0 -2 2 ∞ ∞ ∞ 6 0 3 4 8 5 1 6 0 7 8 1 3 D(3), P(3) D(4), P(4) -4 1 2 -5 nil 1 1 2 1 5 4 nil 1 4 2 1 6 nil nil nil 2 2 4 nil 4 2 1p = nil 3 nil 2 2 p= 4 3 nil 2 1 4 3 4 nil 1 4 3 4 nil 1 nil nil nil 5 nil 4 3 4 5 nil Proiectarea Algoritmilor 2010
24. 24. Example (6) 0 3 -1 4 -4 0 1 -3 2 -4 3 0 -4 1 -1 3 0 -4 1 -1D= 7 4 0 5 3 D= 7 4 0 5 3 2 -1 -5 0 -2 2 -1 -5 0 -2 2 8 5 1 6 0 3 4 8 5 1 6 0 7 8 D(4), P(4) 1 3 D(5), P(5) -4 1 2 -5 nil 1 4 2 1 5 4 nil 3 4 5 1 6 4 nil 4 2 1 4 nil 4 2 1p= 4 3 nil 2 1 p= 4 3 nil 2 1 4 3 4 nil 1 4 3 4 nil 1 4 3 4 5 nil 4 3 4 5 nil Proiectarea Algoritmilor 2010
25. 25. Johnson’s Algorithm We want to find an algorithm that works better than F-W for sparse graphs Use SSSP algorithms  No negative edges: use Dijkstra – n times  (n2*logn)  Problem if we have negative weight edges  cannot use Dijkstra  B-F – n times is not good enough  (n3) Therefore, we want to find an (n2*logn) algorithm that works on both positive and negative weight edges! Combines Bellman-Ford and Dijkstra
26. 26. Johnson’s Algorithm – Idea We would like:  To transform any negative-weighted graph into a graph with positive weights  Such that the minimum path using the new weights is the same path for the original weights (G, W) => (G, W1)  w(u, v) can be negative  w1(u, v) >= 0 for all (u, v)  p is a minimum path u..v in (G, W) => p is also a minimum path u..v in (G, W1)
27. 27. How to Compute W1? Computing W1 is not so simple Simplistic idea:  Find the minimum negative weight edge  Add the absolute value of that weight to all the other weights in the graph  Then, these new weights are always >= 0 3 b 10 b a 8 a 15 5 12 c c d -7 d 0 Look at w(abd) in the two graphs This idea does not work!
28. 28. How to Compute W1? Use B-F New idea: build a new graph G’(V’, E’)  V’ = V U {s}  E’ = E U {(s, v) for all vV}  w’(s, v) = 0 for all vV  w’(u, v) = w(u, v) for all u,vV Run Bellman-Ford on G’ from s  The result is h(v)= δ(s, v) in G’ for all vV  h(v) may be positive or negative  We can also detect the negative cycles in G’ The new weight function for G is:  w1(u, v) = w(u, v) + h(u) – h(v) >= 0 for all (u, v)E
29. 29. Properties of G’ and W1 For any path p = <v0, v1, …, vk> in G:  w1(p) = w(p) + h(v0) – h(vk) The negative weight cycles in G’ are the same as the negative weight cycles in G. Why?  Because the weight of all the cycles is the same for w1 as for w  Cycle p => v0 = vk => w1(p) = w(p) Prove that w1(u, v) = w(u, v) + h(u) – h(v) >= 0 for all (u, v)E  Use triangle inequality for edge (u, v)E  In G’, we have h(v) = δ(s, v) <= h(u) + w(u, v) = δ(s, u) + w(u, v)  Therefore h(u) + w(u, v) – h(v) >= 0
30. 30. Johnson’s Algorithm – PseudocodeJOHNSON(G, W) G’ = (V’,E’); V’ = V  {s}; // add new source s E’ = E  (s,u), uV; w’(s,u) = 0; IF (BF(G’, W’) == FALSE) // run BF on G’ PRINT “Error! Negative cycle was found!” ELSE FOREACH (vV) h(v) = δ(s,v); // computed by Bellman Ford FOREACH ((u,v)E) w1(u,v) = w(u,v) + h(u) - h(v) // compute new positive weights FOREACH (uV) Dijkstra(G,w1,u) // run Dijkstra for each vertex as a source using w1 FOREACH (vV) d(u,v) = δ1(u,v) + h(v) - h(u) // we need to switch back from w1 to wTime complexity: (n*m + n2*logn)  (n2 * logn) for sparse graphs
31. 31. Example (1) 0 -1 2 0 2 3 4 3 4 7 0 0 7 0 8 -5 8 s 1 3 1 3 1 1 2 0 -4 2 -4 -5 -5 5 4 5 4 6 0 6 0 Add s and run -4 B-F on the new graph. Proiectarea Algoritmilor 2010
32. 32. Example (2) 0 -1 0 2 3 40 0 7 0 8 -5 s 1 3 5 -1 1 0 -4 2 1 -5 2 5 4 4 0 0 6 0 0 0 10 -4 0 13 -5 s 1 3 0 4 0 2 0 w1(u,v) = w(u,v) + h(u) - h(v) 5 4 0 2 0 -4 Proiectarea Algoritmilor 2010
33. 33. Example (3) 5 -1 2/1 1 2 2 4 00 0 4 0 10 -5 0/0 2/-3 0 13 10 s 1 3 13 1 3 0 4 0 2 0 0 0 2 0 5 4 0 2 0 5 4 -4 2 0/-4 2/2 Remove s Run Dijkstra from each vertex => (δ1(u, v)). Recompute the distances: d(u,v) = δ1(u,v) + h(v) - h(u) Proiectarea Algoritmilor 2010
34. 34. Example (4) 0/0 0/4 2 2 4 0 4 02/3 10 0/-4 2/7 10 0/0 13 13 1 3 1 3 0 2 0 0 2 0 0 1 -3 2 -4 0 0 5 4 3 0 -4 1 -1 5 4 2 2 2/-1 0/1 7 4 0 5 3 2/3 0/5 0/-1 2 -1 -5 0 -2 2/5 2 2 4 0 8 5 1 6 0 4 0 2/2 10 0/-5 4/8 10 2/1 13 13 1 3 1 3 2 0 2 0 0 0 0 0 5 4 5 4 2 2 2/-2 0/0 0/0 2/6 Proiectarea Algoritmilor 2010
35. 35. Application: Transitive Closure Given a graph G(V, E) Compute the transitive closure of G: G*(V, E*) E* = {(u, v) | there exists at least a path in G from u to v, u..v} G* is an unweighted graph  we only need to compute E* or the adjacency matrix of G* We can use different algorithms One algorithm is an adapted version of Floyd- Warshall  Initialize A(0)[i, j] = 1 if (i, j)E and A(0)[i, j] = 0 o/w  A(k)[i, j] = A(k-1)[i, j] OR (A(k-1)[i, k] AND A(k-1)[k, j])
36. 36. Transitive Closure – PseudocodeTRANSITIVE-CLOSURE(G, W) n = |V[G]| FOR (i = 1; i <= n; i++) FOR (j = 1; j <= n; j++) IF (W[i][j] != INF) A[0][i][j] = 1 ELSE A[0][i][j] = 0 FOR (k = 1; k <= n; k++) FOR (i = 1; i <= n; i++) FOR (j = 1; j <= n; j++) A[k][i][j] = A[k-1][i][j] OR (A[k-1][i][k] AND A[k-1][k][j]) RETURN A[n]
37. 37. Conclusions We can use SSSP algorithms for computing APSP But there are better solutions specific to the APSP problem! Floyd-Warshall for dense graphs: (n3) Johnson for sparse graphs: (n2 * logn)
38. 38. References CLRS – Chapter 25 R. Sedgewick, K Wayne – Algorithms and Data Structures – Princeton 2007 www.cs.princeton.edu/~rs/AlgsDS07/  Problem 1 and the corresponding images are taken from these slides! MIT OCW – Introduction to Algorithms – video lecture 19