Knapsack Problem using Dynamic Programming

1. INFORMATION TECHNOLOGY
(5th SEMESTER)
ANALYSIS AND DESIGN OF ALGORITHM
Guided by:-
Mrs. Kalyani Adawadkar
Prepared by:-
Patel Paras (140500116024)
Patel Pranay (140500116026)
Shah Vidhi (140500116036)
1

2. What is Dynamic Programming?
 Dynamic Programming is typically applied to optimization problem.
• Dynamic programming is technique for solving problems with overlapping
sub problems.
• In this method each sub problem is solved only once. The result of each
sub problem, is recorded in a table from which we can obtain a solution to
the original problem.
2

3. Knapsack Problem
The knapsack problem can be defined as follows:
• If there are n items with the weights w1,w2,…wn and values
(Profit associated with each item) v1,v2,…vn and capacity of
knapsack to be W, then find the most valuable subset of the
items that fit into the knapsack.
3

4. To solve the knapsack problem using dynamic
programming we will write the recurrence relation as :
Max{ table[ i-1 , j }, vi + table[ i-1,j-wi ] } if j ≥ wi
table[ i , j ]= OR
table [ i-1 , j ] if j < w
• That means, a table is constructed using above given formula. Initially ,
Table[0, j] = 0 as well as table [i , 0] = 0 When j ≥ 0 and i ≥ 0.
4

5. The initially stage of the table can be : -
0 0 … 0 … 0 … 0
: Table[i-1,j-wi]
0 Table[i-1,j]
0 Table[i,j]
:
0
Table[n,w]
0
i-1
i
n
0 1 j-wi j w
Goal i.e.,
Maximum
Value of items
• The table [n,W] is a goal i.e., it gives the total items sum of all the selected items for the knapsack.
• From this goal value the selected items can be traced out. Let us solve the knapsack problem, using the
above mentioned formula .
5

6. Example :-
Item Weight Value
1 2 3
2 3 4
3 4 5
4 5 6
The capacity of knapsack is W = 5.
 Obtain optimal solution for the knapsack problem.
6

7.  Initially, table [0,j] = 0 and [I,0] = 0. There are 0 to n rows and 0 to W columns in the
table.
0 0 0 0 0 0
0
0
0
0
0
1
2
3
4
0 1 2 3 4 5
• Now we will fill up the table either row by row or column by column. Let us start filling the table
row by row using following formula :
7

8. Table[1,1] with i=1, j=1, wi=2 and vi =3.
As j < wi ;
Table [1,1] = table [ i-1 , j ]
= table [0,1]
=0
Table[1,2] with i=1, j=2, wi=2 and vi =3.
As j ≥ wi ;
Table [1,2] = max{ table [ i-1 , j ], vi + table[i-1,j-wi] }
= max{ table [0,2] , 3 + table [0,0] }
= max{ 0, 3+0 }
= 3
Table[1,2] with i=1, j=3, wi=2 and vi =3.
As j ≥ wi ;
Table [1,2] = max{ table [ i-1 , j ], vi + table[i-1,j-wi] }
= max{ table [0,2] , 3 + table [0,0] }
= max{ 0, 3+0 }
= 3
0 0 0 0 0 0
0 0 3 3 3 3
0
0
0
0
1
2
3
4
0 1 2 3 4 5
 Compute this values in table
8

9.  Continuing in this fashion we can compute all the values of table.
The table will be -
0 0 0 0 0 0
0 0 3 3 3 3
0 0 3 4 4 7
0 0 3 4 5 7
0 0 3 4 5 7
0
1
2
3
4
0 1 2 3 4 5
7
This is the total
value of selected
items.
9

10. How to find actual items ?
• Now, as we know that table [ n , W ] is the total value of selected items, that
can be placed in the knapsack.
• Following steps are used repeatedly to select actual knapsack item.
Let, I = n and k = W then
While( I > 0 and k > 0)
{
if( table[ i , k ] ≠ table[ i-1 , k ]) then
Mark ith item as in knapsack
i = i - 1 and k = k – wi
else
i=i-1
}
10

11.  Let us apply these steps to the above given problem.
i = 4 and k = 5
i.e., table [4,5] = table [3,5]
.·. Do not select ith i.e., 4th item.
Now set i = i – 1
i = 3
i = 2 and k = 5
i.e., table [2,5] ≠ table [1,5]
.·. Select ith i.e., 2nd item.
Now set i = i – 1
i = 1
and k = k - wi
k = 5 – 3 = 2
i = 3 and k = 5
i.e., table [3,5] = table [2,5]
.·. Do not select ith i.e., 3rd item.
Now set i = i – 1
i = 2
1
2
3 4 i = 1 and k = 52
i.e., table [1,2] ≠ table [0,2]
.·. Select ith i.e., 1st item.
Now set i = i – 1
i = 0
and k = k - wi
k = 2 – 2 = 0
• Thus we have selected item 1 and item 2 for the knapsack.
• This solution can also represented by solution vector (1,1,0,0).
11

12. Algorithm Dynamic_Knapsack(n , W , w[], v[] )
//Pro-Des: Algorithm for obtaining knapsack solution using dynamic programming.
//Input: n is total no. of items, W is the capacity of knapsack , w[] stores weight and v[] store values.
//Output: return total value of selected items for knapsack.
for( i ←0 to n) do
{
for ( j ← 0 to W) do
{
table[i,0]=0
table[0,j]=0
}
}
for ( i ← 0 to n ) do
{
for ( j ← 0 to W ) do
{
if( j <w[ i ] ) then
table[ i , j ] ← table[ i-1 , j ]
else if ( j>= w[ i ]) then
table[ i , j ] ← max ( table [ i-1, j ],( v[i] + table [ i-1, j-w[i] ] ))
}
}
return table[ n , W ]
Algorithm for knapsack problem 12

13. Analysis of knapsack dynamic algorithm :-
• In this algorithm the basic operation is if … else if statement within two for loops.
Hence
c(n)=
𝑗=0
𝑊
1
𝑖=0
𝑛
𝑖=0
𝑛
𝑊 − 0 + 1=
𝑖=0
𝑛
𝑊 + 1
𝑖=0
𝑛
𝑊
𝑖=0
𝑛
1=
=
+
𝑖=0
𝑛
1
𝑖=0
𝑛
1= W +
= W(n-0+1)+(n-0+1)
= Wn + W + n + 1
Thus c(n) ≈ Wn
.·. The time complexity of this algorithm is (nW)
13

14. 14