Knapsack and Edit Distance

1. CS 321. Algorithm Analysis & Design
Lecture 13
Dynamic Programming
Edit Distance
Knapsacks
Shortest Paths

3. Items with weights wi
and value vi

4. Knapsack
with capacity C
Items with weights wi
and value vi

6. w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

7. (What we want)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

8. (What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

9. A[i] stores the best value for the first i items.
(What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

10. A[i] stores the best value for the first i items.
A[i] = max(A(i-1), A[i-1] + vi)
(What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

11. A[i] stores the best value for the first i items.
(What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

12. (What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …

13. A[i,j] stores the best value for the first i items.
(What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …
that fits in a knapsack of size j.

14. A[i,j] stores the best value for the first i items.
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(What we want)
(Build up the array bottom to top.)
w1
v1
w2
v2
wi
vi
wn-1
vn-1
wn
vn
… …
that fits in a knapsack of size j.

15. 1 2 3 4 5
1
2
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)

16. 1 2 3 4 5
1
2
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

17. 1 2 3 4 5
1
2
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

18. 1 2 3 4 5
1
2
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

19. 1 2 3 4 5
1 0 3 3 3 3
2
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

20. 1 2 3 4 5
1 0 3 3 3 3
2
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

21. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

22. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

23. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

24. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3 0 6 7 9 9
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

25. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3 0 6 7 9 9
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

26. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3 0 6 7 9 9
4
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

27. 1 2 3 4 5
1 0 3 3 3 3
2 0 6 6 9 9
3 0 6 7 9 9
4 4 6 10 11 13
A[i,j] = max(A(i-1,j), A[i-1,j-wi] + vi)
(2,3), (2,6), (3,7), (1,4) - weight/value pairs

29. Running time = # of subproblems x time/subproblem

30. Running time = # of subproblems x time/subproblem
nC x O(1) = O(nC)
(n = #items, C = Capacity)

31. Running time = # of subproblems x time/subproblem
nC x O(1) = O(nC)
(n = #items, C = Capacity)
Is this polynomial?

32. Running time = # of subproblems x time/subproblem
nC x O(1) = O(nC)
(n = #items, C = Capacity)
Is this polynomial?
pseudo-polynomial

33. Edit Distance

34. Edit Distance
The cost of changing one word to another,
using insertions/deletions/changes.

35. Edit Distance
The cost of changing one word to another,
using insertions/deletions/changes.
Each operation can have a certain cost.

36. A C G T
A 0 1 1 3
C 2 0 1 1
G 1 2 0 2
T 3 2 1 0

37. SUNNY DEOL
SUNNY LEONE

38. SUNNY DEOL
SUNNY LEONE

39. SUNNY DEOL
SUNNY LEONE
SUNNY LEOL

40. SUNNY DEOL
SUNNY LEONE
SUNNY LEOL
SUNNY LEON

41. SUNNY DEOL
SUNNY LEONE
SUNNY LEOL
SUNNY LEON
SUNNY LEONE

44. Work out the solution for pairs x[i:] and y[j:].

45. (What we want)
(Build up the array bottom to top.)

46. (What we want)
(Build up the array bottom to top.)
c[i,j] stores the edit distance between
x[1…i] and y[1…j].

47. (What we want)
(Build up the array bottom to top.)
c[i,j] stores the edit distance between
x[1…i] and y[1…j].
c[i,j] = min(c[i-1,j] + w1, c[i,j-1] + w2, c[i-1,j-1] + w3)

48. (What we want)
(Build up the array bottom to top.)
c[i,j] stores the edit distance between
x[1…i] and y[1…j].