Upcoming SlideShare
×

# Dynamic Programming

4,384 views
4,347 views

Published on

Algorithm Design lecture notes.

Published in: Education, Technology
5 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
4,384
On SlideShare
0
From Embeds
0
Number of Embeds
75
Actions
Shares
0
270
0
Likes
5
Embeds 0
No embeds

No notes for slide
• overlapping sub-problem = sub-problem whose results can be reused several times
• 1-24,26-37
• A. compiler&apos;s job is easier in pure functional languages since no side effects
• 1-24,26-37
• the term &amp;quot;regression&amp;quot; stems from the work of Sir Francis Galton (1822-1911), a famous geneticist, who studied the heights of fathers and their sons. He found that offspring of a parent of larger than average height tended to be smaller than their parents, and offspring of parents of smaller than average size tended to be larger than their parents. &amp;quot;regression toward mediocrity&amp;quot;. Gauss (1809-1811) used Gaussian elimination to solve multiple linear regression Gauss-Markov theorem (least squares estimates are uniformly minimum variance estimates (UMVUE) among all linear unbiased estimates)
• 1-24,26-37
• ### Dynamic Programming

2. 2. Algorithmic Paradigms <ul><li>Greed. Build up a solution incrementally, myopically optimizing some local criterion. </li></ul><ul><li>Divide-and-conquer. Break up a problem into two sub-problems, solve each sub-problem independently, and combine solution to sub-problems to form solution to original problem. </li></ul><ul><li>Dynamic programming. Carefully decompose the problem into a series of sub-problems, and build up correct solutions to larger and larger sub-problems. </li></ul><ul><li>Dangerously close to the edge of brute force search. </li></ul><ul><li>Systematically works through the set of possible solutions to the problem, it does this without ever examining all of them explicitly. </li></ul><ul><li>It is a tricky technique and may need time to get used to. </li></ul>
3. 3. Dynamic Programming History <ul><li>Bellman. Pioneered the systematic study of dynamic programming in the 1950s. </li></ul>Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography.
4. 4. Dynamic Programming Applications <ul><li>Areas. </li></ul><ul><ul><li>Bioinformatics. </li></ul></ul><ul><ul><li>Control theory. </li></ul></ul><ul><ul><li>Information theory. </li></ul></ul><ul><ul><li>Operations research. </li></ul></ul><ul><ul><li>Computer science: theory, graphics, AI, systems, …. </li></ul></ul><ul><li>Some famous dynamic programming algorithms. </li></ul><ul><ul><li>Viterbi for hidden Markov models. </li></ul></ul><ul><ul><li>Unix diff for comparing two files. </li></ul></ul><ul><ul><li>Smith-Waterman for sequence alignment. </li></ul></ul><ul><ul><li>Bellman-Ford for shortest path routing in networks. </li></ul></ul><ul><ul><li>Cocke-Kasami-Younger for parsing context free grammars. </li></ul></ul>
5. 5. 6.1 Weighted Interval Scheduling
6. 6. Weighted Interval Scheduling <ul><li>Weighted interval scheduling problem. </li></ul><ul><ul><li>Job j starts at s j , finishes at f j , and has weight or value v j . </li></ul></ul><ul><ul><li>Two jobs compatible if they don't overlap. </li></ul></ul><ul><ul><li>Goal: find maximum weight subset of mutually compatible jobs. </li></ul></ul>Time 0 1 2 3 4 5 6 7 8 9 10 11 f g h e a b c d
7. 7. Unweighted Interval Scheduling Review <ul><li>Recall. Greedy algorithm works if all weights are 1. </li></ul><ul><ul><li>Consider jobs in ascending order of finish time. </li></ul></ul><ul><ul><li>Add job to subset if it is compatible with previously chosen jobs. </li></ul></ul><ul><li>Observation. Greedy algorithm can fail spectacularly if arbitrary weights are allowed. </li></ul>Time 0 1 2 3 4 5 6 7 8 9 10 11 b a weight = 999 weight = 1
8. 8. Weighted Interval Scheduling <ul><li>Notation. Label jobs by finishing time: f 1  f 2  . . .  f n . </li></ul><ul><li>Def. p(j) = largest index i < j such that job i is compatible with j. </li></ul><ul><li>Ex: p(8) = 5, p(7) = 3, p(2) = 0. </li></ul>Time 0 1 2 3 4 5 6 7 8 9 10 11 6 7 8 4 3 1 2 5
9. 9. Dynamic Programming: Binary Choice <ul><li>Notation. OPT(j) = value of optimal solution to the problem consisting of job requests 1, 2, ..., j. </li></ul><ul><ul><li>Case 1: OPT selects job j. </li></ul></ul><ul><ul><ul><li>can't use incompatible jobs { p(j) + 1, p(j) + 2, ..., j - 1 } </li></ul></ul></ul><ul><ul><ul><li>must include optimal solution to problem consisting of remaining compatible jobs 1, 2, ..., p(j) </li></ul></ul></ul><ul><ul><li>Case 2: OPT does not select job j. </li></ul></ul><ul><ul><ul><li>must include optimal solution to problem consisting of remaining compatible jobs 1, 2, ..., j-1 </li></ul></ul></ul>optimal substructure
10. 10. Weighted Interval Scheduling: Brute Force <ul><li>Brute force algorithm. </li></ul>Input : n, s 1 ,…,s n , f 1 ,…,f n , v 1 ,…,v n Sort jobs by finish times so that f 1  f 2  ...  f n . Compute p(1), p(2), …, p(n) Compute-Opt(j) { if (j = 0) return 0 else return max(v j + Compute-Opt(p(j)), Compute-Opt(j-1)) } Running time: T(n) = T(p(n)) + T(n - 1) + O(1) =…
11. 11. Weighted Interval Scheduling: Brute Force <ul><li>Observation. Recursive algorithm fails spectacularly because of redundant sub-problems  exponential algorithms. </li></ul><ul><li>Ex. Number of recursive calls for family of &quot;layered&quot; instances grows like Fibonacci sequence. </li></ul>3 4 5 1 2 p(1) = 0, p(j) = j-2 Running time: T(n) = T(p(n)) + T(n - 1) + O(1) =T(n-2)+T(n-1)+O(1)= θ ( φ n ) where φ = 1:618 is the golden ratio 5 4 3 3 2 2 1 2 1 1 0 1 0 1 0
12. 12. Weighted Interval Scheduling: Memoization <ul><li>Memoization. Store results of each sub-problem in a cache; lookup as needed. </li></ul>Input : n, s 1 ,…,s n , f 1 ,…,f n , v 1 ,…,v n Sort jobs by finish times so that f 1  f 2  ...  f n . Compute p(1), p(2), …, p(n) for j = 1 to n M[j] = empty M[j] = 0 M-Compute-Opt(j) { if (M[j] is empty) M[j] = max(w j + M-Compute-Opt(p(j)), M-Compute-Opt(j-1)) return M[j] } global array
13. 13. Weighted Interval Scheduling: Running Time <ul><li>Claim. Memoized version of algorithm takes O(n log n) time. </li></ul><ul><ul><li>Sort by finish time: O(n log n). </li></ul></ul><ul><ul><li>Computing p(  ) : O(n) after sorting by start time. </li></ul></ul><ul><ul><li>M-Compute-Opt(j) : each invocation takes O(1) time and either </li></ul></ul><ul><ul><ul><li>(i) returns an existing value M[j] </li></ul></ul></ul><ul><ul><ul><li>(ii) fills in one new entry M[j] and makes two recursive calls </li></ul></ul></ul><ul><ul><li>Progress measure  = # nonempty entries of M[] . </li></ul></ul><ul><ul><ul><li>initially  = 0, throughout   n. </li></ul></ul></ul><ul><ul><ul><li>(ii) increases  by 1  at most 2n recursive calls , since there are only n entries to fill . </li></ul></ul></ul><ul><ul><li>Overall running time of M-Compute-Opt(n) is O(n). ▪ </li></ul></ul><ul><li>Remark. O(n) if jobs are pre-sorted by start and finish times. </li></ul>
14. 14. Automated Memoization <ul><li>Automated memoization. Many functional programming languages (e.g., Lisp) have built-in support for memoization. </li></ul><ul><li>Q. Why not in imperative languages (e.g., Java)? </li></ul><ul><li>A. compiler's job is easier in pure functional languages since no side effects </li></ul>static int F(int n) { if (n <= 1) return n; else return F(n-1) + F(n-2); } (defun F (n) (if (<= n 1) n (+ (F (- n 1)) (F (- n 2))))) Lisp (efficient) Java (exponential) F(40) F(39) F(38) F(38) F(37) F(36) F(37) F(36) F(35) F(36) F(35) F(34) F(37) F(36) F(35)
15. 15. Weighted Interval Scheduling: Finding a Solution <ul><li>Q. Dynamic programming algorithms computes optimal value. What if we want the solution itself? </li></ul><ul><li>A. Do some post-processing. </li></ul><ul><ul><li># of recursive calls  n  O(n). </li></ul></ul>Run M-Compute-Opt(n) Run Find-Solution(n) Find-Solution(j) { if (j = 0) output nothing else if (v j + M[p(j)] > M[j-1]) print j Find-Solution(p(j)) else Find-Solution(j-1) }
16. 16. Weighted Interval Scheduling: Bottom-Up <ul><li>Bottom-up dynamic programming. Unwind recursion. </li></ul>Input : n, s 1 ,…,s n , f 1 ,…,f n , v 1 ,…,v n Sort jobs by finish times so that f 1  f 2  ...  f n . Compute p(1), p(2), …, p(n) Iterative-Compute-Opt { M[0] = 0 for j = 1 to n M[j] = max(v j + M[p(j)], M[j-1]) }
17. 17. Dynamic Programming Overview <ul><li>Dynamic Programming = Recursion + Memoization </li></ul><ul><li>1 Formulate problem recursively in terms of solutions to polynomially many sub-problems </li></ul><ul><li>2 Solve sub-problems bottom-up, storing optimal solutions </li></ul>
18. 18. 6.3 Segmented Least Squares
19. 19. Segmented Least Squares <ul><li>Least squares. </li></ul><ul><ul><li>Foundational problem in statistic and numerical analysis. </li></ul></ul><ul><ul><li>Given n points in the plane: (x 1 , y 1 ), (x 2 , y 2 ) , . . . , (x n , y n ). </li></ul></ul><ul><ul><li>Find a line y = ax + b that minimizes the sum of the squared error: </li></ul></ul><ul><li>Solution. Calculus  min error is achieved when </li></ul>x y
20. 20. Segmented Least Squares <ul><li>Segmented least squares. </li></ul><ul><ul><li>Points lie roughly on a sequence of several line segments. </li></ul></ul><ul><ul><li>Given n points in the plane (x 1 , y 1 ), (x 2 , y 2 ) , . . . , (x n , y n ) with </li></ul></ul><ul><ul><li>x 1 < x 2 < ... < x n , find a sequence of lines that minimizes f(x) . </li></ul></ul><ul><li>Q. What's a reasonable choice for f(x) to balance accuracy and parsimony? </li></ul>x y goodness of fit number of lines
21. 21. Segmented Least Squares <ul><li>Segmented least squares. </li></ul><ul><ul><li>Points lie roughly on a sequence of several line segments. </li></ul></ul><ul><ul><li>Given n points in the plane (x 1 , y 1 ), (x 2 , y 2 ) , . . . , (x n , y n ) with </li></ul></ul><ul><ul><li>x 1 < x 2 < ... < x n , find a sequence of lines that minimizes: </li></ul></ul><ul><ul><ul><li>the sum of the sums of the squared errors E in each segment </li></ul></ul></ul><ul><ul><ul><li>the number of lines L </li></ul></ul></ul><ul><ul><li>Tradeoff function: E + c L, for some constant c > 0. </li></ul></ul>x y
22. 22. Structure of Optimal Solution <ul><li>Suppose the last point p n = (x n ; y n ) is part of a segment that starts at pi = (x i ; y i ) </li></ul><ul><li>Then optimal solution is </li></ul><ul><li>optimal solution for { p 1 … p i - 1 } </li></ul><ul><li>plus </li></ul><ul><li>(best) line through { p i … p n } </li></ul><ul><li>Let: </li></ul><ul><li>Opt(j) : the cost of the fir st j points </li></ul><ul><li>e(j ; k) th e error of the best line through points j to k </li></ul><ul><li>Then: </li></ul><ul><li>Opt(n) = e(i ; n) + C + Opt(i - 1) </li></ul>
23. 23. Dynamic Programming: Multiway Choice <ul><li>Notation. </li></ul><ul><ul><li>OPT( j ) = minimum cost for points p 1 , … p i p i+1 , . . . , p j . </li></ul></ul><ul><ul><li>e(i, j) = minimum sum of squares for points p i , p i+1 , . . . , p j . </li></ul></ul><ul><li>To compute OPT(j): </li></ul><ul><ul><li>Last segment uses points p i , p i+1 , . . . , p j for some i. </li></ul></ul><ul><ul><li>Cost = e(i, j) + c + OPT(i-1). </li></ul></ul>x y p i …… p j
24. 24. Segmented Least Squares: Algorithm <ul><li>Running time. O(n 3 ). </li></ul><ul><ul><li>Bottleneck = computing e(i, j) for O(n 2 ) pairs, O(n) per pair using previous formula. </li></ul></ul>INPUT : n, p 1 ,…,p N , c Segmented-Least-Squares() { M[0] = 0 for j = 1 to n for i = 1 to j compute the least square error e ij for the segment p i ,…, p j for j = 1 to n M[j] = min 1  i  j (e ij + c + M[i-1]) return M[n] } can be improved to O(n 2 ) by pre-computing various statistics
25. 25. 6.4 Knapsack Problem
26. 26. Knapsack Problem <ul><li>Knapsack problem. </li></ul><ul><ul><li>Given n objects and a &quot;knapsack.&quot; </li></ul></ul><ul><ul><li>Item i weighs w i > 0 kilograms and has value v i > 0. </li></ul></ul><ul><ul><li>Knapsack has capacity of W kilograms. </li></ul></ul><ul><ul><li>Goal: fill knapsack so as to maximize total value. </li></ul></ul><ul><li>Ex: { 3, 4 } has value 40. </li></ul><ul><li>Greedy: repeatedly add item with maximum ratio v i / w i . </li></ul><ul><li>Ex: { 5, 2, 1 } achieves only value = 35  greedy not optimal. </li></ul>1 Value 18 22 28 1 Weight 5 6 6 2 7 Item 1 3 4 5 2 W = 11
27. 27. Dynamic Programming: False Start <ul><li>Def. OPT(i) = max profit subset of items 1, …, i. </li></ul><ul><ul><li>Case 1: OPT does not select item i. </li></ul></ul><ul><ul><ul><li>OPT selects best of { 1, 2, …, i-1 } </li></ul></ul></ul><ul><ul><li>Case 2: OPT selects item i. </li></ul></ul><ul><ul><ul><li>accepting item i does not immediately imply that we will have to reject other items </li></ul></ul></ul><ul><ul><ul><li>without knowing what other items were selected before i, we don't even know if we have enough room for i </li></ul></ul></ul><ul><li>Conclusion. Need more sub-problems! </li></ul>
28. 28. Dynamic Programming: Adding a New Variable <ul><li>Def. OPT(i, w) = max profit subset of items 1, …, i with weight limit w. </li></ul><ul><ul><li>Case 1: OPT does not select item i. </li></ul></ul><ul><ul><ul><li>OPT selects best of { 1, 2, …, i-1 } using weight limit w </li></ul></ul></ul><ul><ul><li>Case 2: OPT selects item i. </li></ul></ul><ul><ul><ul><li>new weight limit = w – w i </li></ul></ul></ul><ul><ul><ul><li>OPT selects best of { 1, 2, …, i–1 } using this new weight limit </li></ul></ul></ul>
29. 29. Knapsack Problem: Bottom-Up <ul><li>Knapsack. Fill up an n-by-W array. </li></ul>Input : n, w 1 ,…,w N, v 1 ,…,v N for w = 0 to W M[0, w] = 0 for i = 1 to n for w = 1 to W if (w i > w) M[i, w] = M[i-1, w] else M[i, w] = max {M[i-1, w], v i + M[i-1, w-w i ]} return M[n, W]
30. 30. Knapsack Algorithm n + 1 W + 1 W = 11 OPT: { 4, 3 } value = 22 + 18 = 40 1 Value 18 22 28 1 Weight 5 6 6 2 7 Item 1 3 4 5 2  { 1, 2 } { 1, 2, 3 } { 1, 2, 3, 4 } { 1 } { 1, 2, 3, 4, 5 } 0 0 0 0 0 0 0 1 0 1 1 1 1 1 2 0 6 6 6 1 6 3 0 7 7 7 1 7 4 0 7 7 7 1 7 5 0 7 18 18 1 18 6 0 7 19 22 1 22 7 0 7 24 24 1 28 8 0 7 25 28 1 29 9 0 7 25 29 1 34 10 0 7 25 29 1 34 11 0 7 25 40 1 40
31. 31. Knapsack Problem: Running Time <ul><li>Running time.  (n W). </li></ul><ul><ul><li>Not polynomial in input size! </li></ul></ul><ul><ul><li>&quot;Pseudo-polynomial.&quot; </li></ul></ul><ul><ul><li>Decision version of Knapsack is NP-complete. [Chapter 8] </li></ul></ul><ul><li>Knapsack approximation algorithm. There exists a polynomial algorithm that produces a feasible solution that has value within 0.01% of optimum. [Section 11.8] </li></ul>