Greedy Knapsack Kruskal Prim Job Sequencing with Deadlines Dijkstra's algorithm

1. MODULE 3
The Greedy Method
SHIWANI GUPTA 1

2. GREEDY APPROACH
• For an optimization problem, we are given a set of constraints and an
optimization function.
• Solutions that satisfy the problem constraints are called feasible
solutions.
• A feasible solution for which the optimization function has the best
possible value is called an optimal solution.
• A “greedy algorithm” sometimes works well for optimization
problems. It works in phases:
• You take the best you can get right now, without regard for future
consequences
• You hope that by choosing a local optimum at each step, you will
end up at a global optimum
• Once made, the choice can’t be changed on subsequent steps
(irrevocable).
“Greedy algorithms do not always yield optimal solutions.”
SHIWANI GUPTA 2

3. Applications of Greedy Method
 Knapsack problem
 Minimum Spanning Tree
 Kruskal’s Algorithm
 Prim’s Algorithm
 Job sequencing with deadlines
 Finding Shortest path
 Dijkstra’s Algorithm
 Optimal Storage on Tapes
SHIWANI GUPTA 3

4. Knapsack Problem
 You have a knapsack that has capacity (weight) C.
 You have several items I1,…,In.
 Each item Ij has a weight wj and a benefit bj.
 You want to place a certain number of copies of each item Ij in the
knapsack so that:
 The knapsack weight capacity is not exceeded and
 The total benefit is maximal.
SHIWANI GUPTA 4

5. Knapsack Problem Variants
 0/1 Knapsack problem: Similar to the knapsack problem except that
for each item, only 1 copy is available (not an unlimited number as we
have been assuming so far).
 the items cannot be divided
 one must take entire item or leave it behind
 Fractional knapsack problem: You can take a fractional number of
items. Has the same constraint as 0/1 knapsack. Can solve using a
greedy algorithm.
 one can take partial items
 for instance, items are liquids or powder
SHIWANI GUPTA 5

6. The Fractional Knapsack Problem
 Given: A set S of n items, with each item i having
 bi - a positive benefit
 wi - a positive weight
 Goal: Choose items with maximum total benefit but with weight at
most W.
 If we are allowed to take fractional amounts, then this is the
fractional knapsack problem.
 In this case, we let xi denote the amount we take of item i
 Objective: maximize
 Constraint:

S
i
i
i x
b



S
i
i
i W
w
x
SHIWANI GUPTA 6

7. Fractional knapsack
 For item Ij, let rj = bj/wj. This gives you the benefit per measure of
weight.
 Sort the items in descending order of rj
 Pack the knapsack by putting as many of each item as you can
walking down the sorted list.
 Given positive integers P1, P2, …, Pn, W1, W2, …, Wn and M.
 Find X1, X2, … ,Xn, 0≦Xi≦1 such that is maximized.
 Subject to constraint


n
1
i
i
iX
P



n
1
i
i
i M
X
W
SHIWANI GUPTA 7

8. Knapsack Problem Example
 M = 20, (P1, P2, P3) = (25,24,15)
(W1, W2, W3) = (18, 15, 10)
 Four feasible solutions, 4 is optimal
(X1, X2, X3) ΣWiXi ΣPiX
1. (1/2,1/3,1/4) 16.5 24.25
2. (1,2/15,0) 20 28.2
3. (0, 2/3, 1) 20 31
4. (0, 1, 1/2) 20 31.5
Sol. 2: Greedy strategy using total profit as optimization function:-
Suboptimal
Sol. 3: Greedy strategy using weight (capacity used) as optimization
function:- Suboptimal
Sol. 4: Greedy strategy using ratio of profit to weight (pi/wi) as
optimization function :- Optimal SHIWANI GUPTA 8

9. The knapsack algorithm
 The greedy algorithm:
Step 1: Sort pi/wi into nonincreasing order.
Step 2: Put the objects into the knapsack according
to the sorted sequence as possible as we can.
 e. g.
n = 3, M = 20, (p1, p2, p3) = (25, 24, 15)
(w1, w2, w3) = (18, 15, 10)
Sol: p1/w1 = 25/18 = 1.32
p2/w2 = 24/15 = 1.6
p3/w3 = 15/10 = 1.5
Optimal solution: x1 = 0, x2 = 1, x3 = 1/2
total profit = 24 + 7.5 = 31.5
SHIWANI GUPTA 9

10. Algorithm greedyKnapsack(m, n)
// p[1:n], w[1:n] is profit and wt. of n objects created; all positive
// such that p[i]/w[i]>=p[i+1]/w[i+1] ; m>0 is knapsack size,
// x[1:n] is sol vector that maximizes total benefit without exceeding
//capacity
for each item i in S
x[i]  0.0 //initialize
v[i]=p[i]/w[i]
w=m //remaining capacity in knapsack
for each item i in S
if (w[i]<=w)
x[i]=1.0
w-=w[i]
else if (i<=n)
x[i]=w/w[i]
w-=w[i]*x[i]
The knapsack algorithm
Time Complexity : O(n logn)
: O(n)
SHIWANI GUPTA 10

11. Item Weight Value
A 90 30
B 50 50
C 20 70
D 35 20
Knapsack Capacity : 100
Item Weight Value
A 5 30
B 10 50
C 15 60
Knapsack Capacity : 25
SHIWANI GUPTA 11

12. MINIMUM SPANNING TREES
 Let G = (V,E) be an un-directed connected graph.
 A sub-graph T = (V1, E1) of G is a spanning tree of G iff T is a tree.
O O O O O O O O
O O O O O O O
(a) (b) (c) (d)
(a) Is a complete graph
(b),(c),(d) are three of A’s spanning trees.
(A minimal connected sub-graph of G which includes all the vertices of
G is a spanning tree of G)
SHIWANI GUPTA 12

13. Minimum Spanning Tree
find subset of edges
– that span all the nodes
– create no cycle
– minimize sum of weights
 There can be many spanning trees of a graph
 In fact, there can be many minimum spanning trees of a graph
 But if every edge has a unique weight, then there is a unique MST
Application of MST:
• Designing efficient routing algorithms
• Network Design
• Cable to connect computers
• Obtain independent set of circuit equations for an electrical network
SHIWANI GUPTA 13

14. Kruskal's MST algorithm
7
16
4
5
6
8
11
15
14
17
10
13
3
12
2
9
18
The edges are considered in the non decreasing order.
Kruskal's algorithm maintains a forest that grows until it forms a
spanning tree
SHIWANI GUPTA 14

15. KRUSKAL’S ALGORITHM
Procedure Kruskal (E, cost, n, T, mincost)
// E is the set of edges in G and G has n vertices
// cost (u, v) is the cost of edge (u , v)
// T is the set of edges in the minimum cost spanning tree and
// mincost is the cost
mincost, cost(1: n, 1:n)
parent, T(1:n-1, 2), n
parent-1 //Each vertex is in the different set
i mincost 0
while (i < n) and (heap not empty) do
delete a minimum cost edge (u,v) from the heap
and reheapify using ADJUST
jFIND (u); kFIND (v)
if j ≠ k then //check for cycle
ii+1
T(i, 1)  u;
T (i,2) v SHIWANI GUPTA 15

16. mincost  mincost + cost(u,v)
Call union (u, v)
endif
repeat if i ≠ n-1 // heap is empty but i ≠ n-1
then “no spanning tree”
endif
return
END KRUSKAL
Time complexity of Kruskal’s: O(|E| log|E|)
Theroem: Kruskal’s algorithm generates a minimum cost spanning tree.
SHIWANI GUPTA 16

17. An example of Kruskal’s
algorithm
SHIWANI GUPTA 17

18. Example
Initially each vertex is in a different set {1}{2}{3}{4}{5}{6}
Consider(1, 2); j = 1 = Find(1); k = 2 = Find(2); 1 ≠2 so i1
T(1)={1,2}
(1,2) is included and union (1,2) = {1,2} mincost=10
SHIWANI GUPTA 18

19. Example
Consider (6,3); Test if (6,3) is forming a cycle with (1,2)
6 Find (6); 3 Find (3); 3 ≠ 6 so i2 T(2)={3,6}
(3,6) is included and union (6,3)= {1,2,3,6} mincost=10+15
SHIWANI GUPTA 19

20. Example
Consider (6,4); Test if (6,4) is forming a cycle 3 Find(6) ; 4
Find(4);
3≠ 4 so i 3 and (6,4) is included and union (6, 4) = {1,2,3,4,6}
T(2)={3,6} U {6,4}={3,4,6} mincost=10+15+20
SHIWANI GUPTA 20

21. Example
Consider (6,2); Test if (6,2) is forming a cycle 3 Find(6) ; 1
Find(2) T(1)=T(1) U T(2)={1,2,3,4,6}
mincost=10+15+20+25
1 ≠ 3 so i 4 and (6,2) is added and union (2, 6) = {1,2,3,4,6}
SHIWANI GUPTA 21

22. Example contd…
Consider (1,4); Test if (1,4) is forming a cycle 2  Find (1); 6  Find
(4) AND 2  FIND(6) Reject(since in same set)
Consider (5,3); Test if (5,3) is forming a cycle 5 Find(5) ; 6
Find(3)
5 ≠ 6 so i 5 and (5,3) is added and union (3, 5) = {1,2,3,4,5,6}
mincost=10+15+20+25+35=105 T(1)= {1,2,3,4,6} U
{5,3}={1,2,3,4,5,6}
SHIWANI GUPTA 22

23. Example contd…
Consider(5,2 ); Test if (5,2) is forming a cycle 3 Find (5); 6 Find (2)
Reject(since in same set)
Consider(1, 5 ); Test if (1,5) is forming a cycle 2 Find (1); 3 Find
(5) Reject(since in same set)
Consider(2,3 ); Test if (2,3) is forming a cycle 6 Find (2); 5 Find (3)
Reject(since in same set)
Consider(5,6); Test if (5,6) is forming a cycle 3 Find (5); 2 Find (6)
Reject(since in same set)
Stop since all edges checked
Min Cost is 105
SHIWANI GUPTA 23

24. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 24

25. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
(a)
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 25

26. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
10
(b)
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 26

27. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
10
(c)
12
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 27

28. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
10
12
(d)
14
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 28

29. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
10
16
12
(e)
14
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 29

30. Kruskal’s Algorithm Eg.2
1
2
7
6 3
5
4
10
14 16
22
12
(f)
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12
SHIWANI GUPTA 30

31. Kruskal’s Algorithm Eg.2
1
2
7 3
5
4
10
14 16
22
12
(g)
6
25
SHIWANI GUPTA 31
1
2
7
6 3
5
4
10
28
14 16
25
24 18
22
12

32. Kruskal Example 3
SHIWANI GUPTA 32

33. Kruskal Example 3
SHIWANI GUPTA 33

34. Kruskal Example 3
SHIWANI GUPTA 34

35. Prim’s MST Algorithm
 If A is the set of edges selected so far, then A forms a tree.
 The next edges (u,v) to be included in A is a minimum cost edge not
in A with the property that A incuding v; {u, v} is also a tree.
 Keep just one tree and grow it until it spans all the nodes.
 At each iteration, choose the minimum weight outgoing edge to add
7
16
4
5
6
8
11
15
14
17
10
13
3
12
2
9
18
SHIWANI GUPTA 35

36. PRIM’S ALGORITHM
Procedure PRIM (E, cost, n, T, mincost)
// E is the set of edges in G
// cost (n,n) is the adjacency matrix such that cost (i,j) is a +ve real no
// or cost (i,j) is ∞ if no edge (i,j) exists
// A minimum cost spanning tree is computed and stored as a set
// of edges in the array T(1:n-1,2) where
// (T(i,1), T(i,2)) is an edge in minimum cost spanning tree
cost (n,n), mincost;
near(n), i, j, k, l, T(1:n-1,2)
(k,l)  edges with minimum cost //O(|E|)
mincost  cost (k,l) //Ѳ (1)
(T(1,1), T(1,2) )  (k,l) //tree comprises only of edge (k,l)
for i  1 to n do // building tree edge by edge
if cost (i,l) < cost (i,k) then
near (i)  l
else
near (i)  k SHIWANI GUPTA 36

37. endif
near(k)  near (l)  0 // (k, l) is already in the tree
for i  2 to n-1 do // find n-2 additional edges
for T
Let j be an index such that //select (j, near(j)) as next edge
near (j) ≠ 0 and cost (j, near(j)) is minimum //j in tree
(T(i,1),T(i,2))  (j, near (j))
mincost  mincost + cost (j, near (j))
near (j)  0
for k  1 to n do // update near
if near (k) ≠ 0 and cost(k, near (k)) > cost(k, j)
then near (k)  j
endif
if mincost > ∞ then print (‘no spanning tree’)
END PRIM
O(n2), n = |V|.
Time complexity of Prim’s:
SHIWANI GUPTA 37

38. Example 1
Minimum cost edge (1, 2) with cost 10 is included
near (3)  2 near (4) 1
near (5) 2 near (6) 2
near (5)  1
Select out of 3,4,5,6 a vertex such that
{Cost (3, 2) = 50
Cost (4, 1) = 30
Cost (5, 2) = 40
Cost (6, 2) = 25} is minimum; it is 6 j6
so the edge (j, near (j)) i.e. (6, 2) is included.
Now let us update near (k) values k =1…6
near (1) = near (2) = near (6) = 0
k=3; cost (3, near (3) = 2) = 50 > cost (3, 6) =15
 near (3) is 6; cost(3,6)=15
k=4; near (4)  6  cost (4,6) = 20 = cost (4,6) = 20
k=5; near (5) 2  cost (5,2) = 40 < cost (5,6) = 55
SHIWANI GUPTA 38

39. Example 1 contd…
cost (6,3) is included
Now let us update near (k) values k =1…6
near (1) = near (2) = near (6) = near(3) = 0
k=4; cost (4, near (4) = 6) = 20
k=5; near (5) 3  cost (5,3) = 35
cost (6,4) is included
Now let us update near (k) values k =1…6
near (1) = near (2) = near (6) = near(3) = near(4) = 0
K=5; cost(5,near (5) = 3) = 35
cost (5,3) is included
near (1) = near (2) = near (6) = near(3) = near(4) = near(5) = 0
Stop since all vertices checked
Min Cost is 105
SHIWANI GUPTA 39

40. Prim’s Algorithm Example 2
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 40

41. Prim’s Algorithm Example 2
1
2
7
6 3
5
4
10
(a)
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 41

42. Prim’s Algorithm Example 2
1
2
7
6 3
5
4
10
25
(b)
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 42

43. Prim’s Algorithm Example 2
1
2
7
6
3
5
4
10
25
22
(c)
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 43

44. Prim’s Algorithm Example 2
1
2
7
6 3
5
4
10
25
22
12
(d)
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 44

45. Prim’s Algorithm Example 2
1
2
7
6 3
5
4
10
16
25
22
12
(e)
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 45

46. Prim’s Algorithm Example 2
1
2
7
6 3
5
4
10
14 16
25
22
12
(f)
1
2
7
6 3
5
4
10
28
14 16
25
24
18
22
12
SHIWANI GUPTA 46

47. Example 3 for Prim’s algorithm
SHIWANI GUPTA 47

48. Prim Example 4
SHIWANI GUPTA 48

49. Prim Example
SHIWANI GUPTA 49

50. Job Sequencing with Deadlines
 Given n jobs. Associated with job i is an integer deadline di≧0.
 For any job i the profit pi is earned iff the job is completed by its
deadline. To complete a job, one has to process the job on a machine
for one unit of time.
 A feasible solution is a subset j of jobs such that each job in the subset
can be completed by its deadline. We want to maximize the
 Consider all possible schedules and compute minimum total time in
the system
JP
i i
Time complexity : O (n2)
SHIWANI GUPTA 50

51. n = 4, (p1, p2, p3, p4) = (100,10,15,27)
(d1, d2, d3, d4) = (2, 1, 2, 1)
Feasible solution Processing sequence value
1 (1) 1 100
2 (2) 2 10
3 (3) 3 15
4 (4) 4 27
5 (1,2) 2,1 110
6 (1,4) 4,1 127
7 (2,3) 2,3 25
8 (3,4) 4,3 42
SHIWANI GUPTA 51

52. Example
n = 4, (p1, p2, p3, p4) = (70, 12, 18, 35)
(d1, d2, d3, d4) = (2, 1, 2, 1)
n = 5, (p1, p2, p3, p4) = (20, 15, 10, 5, 1)
(d1, d2, d3, d4) = (2, 2, 1, 3, 3)
SHIWANI GUPTA 52

53. TASK
SHIWANI GUPTA 4 -53
Qs on slide 11 as Slide 8 for Knapsack
Eg. 24, 32 as slide 22, 23 for Kruskal
Eg. 40, 47, 48 as slide 38, 39 for Prim
Qs on Slide 52 as slide 51 for JSD

54. Weighted Graphs
 In a weighted graph, each edge has an associated numerical value,
called the weight of the edge
 Edge weights may represent distances, costs, etc.
 Example:
 In a flight route graph, the weight of an edge represents the
distance in miles between the endpoint airports
ORD
PVD
MIA
DFW
SFO
LAX
LGA
HNL
SHIWANI GUPTA 54

55. Shortest Path Problem
 Given a weighted graph and two vertices u and v, we want to find a
path of minimum total weight between u and v.
 Length of a path is the sum of the weights of its edges.
 Applications
 Internet packet routing
 Flight reservations
 Driving directions
 Djikstra’s algorithm
 Find shortest paths from source s to all other destinations
SHIWANI GUPTA 55

56. Execution of Dijkstra’s algorithm
Iteration N D2 D3 D4 D5 D6
Initial {1} 3 2 5  
1 {1,3} 3 2 4  3
2 {1,2,3} 3 2 4 7 3
3 {1,2,3,6} 3 2 4 5 3
4 {1,2,3,4,6} 3 2 4 5 3
5 {1,2,3,4,5,6} 3 2 4 5 3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3 1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3 1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3 1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3









SHIWANI GUPTA 56

57. Shortest Paths in Dijkstra’s Algorithm
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3 3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3 1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3
1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3 1
2
4
5
6
1
1
2
3
2
3
5
2
4
3
3
SHIWANI GUPTA 57

58. Dijkstra’s algorithm
Algorithm SSSP(p,cost,Dist,n)
{
//cost[1:n,1:n] is an adjacency matrix storing cost of each edge
//Dist [1:n] is a set storing shortest path from source ‘p’ to any other
//vertex
//S, a boolean array stores all visited vertices
for i  1 to n do
{
S[i]  0
Dist [i] cost[p,i]
}
S[p]  1 //put p in S
Dist[p] = 0.0
SHIWANI GUPTA 58

59. for val  2 to n-2 do
{ //obtain n-1 paths from p
Dist[q]=min{Dist[i]} //q chosen from unvisited vertices with
//min dist
S[q]=1
/*update distance value of other nodes*/
for (all node r adjacent to q with S[r]=0) do
if (Dist[r]>(Dist[q]+cost[p,q])) then
Dist[r]  Dist[q] + cost[p,q]
}
}
Dijkstra’s algorithm contd.
Time complexity : O(n2)
SHIWANI GUPTA 59
https://www.geeksforgeeks.org/dijkstras-algorithm-for-adjacency-list-representation-greedy-algo-8/

60. The single-source shortest path problem
 shortest paths from v0 to all destinations
SHIWANI GUPTA 60

61. Dijkstra’s algorithm
1 2 3 4 5 6 7 8
1 0
2 300 0
3 1000 800 0
4 1200 0
5 1500 0 250
6 1000 0 900 1400
7 0 1000
8 1700 0
Cost adjacency matrix.
All entries not shown
are +.
SHIWANI GUPTA 61

62. Vertex
Iteration S Selected (1) (2) (3) (4) (5) (6) (7) (8)
Initial ----
1 5 6 + + + 1500 0 250 + +
2 5,6 7 + + + 1250 0 250 1150 1650
3 5,6,7 4 + + + 1250 0 250 1150 1650
4 5,6,7,4 8 + + 2450 1250 0 250 1150 1650
5 5,6,7,4,8 3 3350 + 2450 1250 0 250 1150 1650
6 5,6,7,4,8,3 2 3350 3250 2450 1250 0 250 1150 1650
5,6,7,4,8,3,2 3350 3250 2450 1250 0 250 1150 1650
SHIWANI GUPTA 62

63. TASK
SHIWANI GUPTA 63
Qs on slide 11 as Slide 8 for Knapsack
Eg. 24, 32 as slide 22, 23 for Kruskal
Eg. 40, 47, 48 as slide 38, 39 for Prim
Qs on Slide 52 as slide 51 for JSD
Qs on Slide 59, 60 as Slide 55, 56 for Dijkstra