week 9 lec 15.pptx

CSC354-Design and Analysis of
Algorithms
Week-9 Lecture-15
Semester#4 Spring 2022
Prepared by:
Adeel Munawar
1

Instructor Contact Details
 Name: Adeel Munawar
 Course Instructor: Design and Analysis of Algorithms
 Credit Hours: 3
 Office Location: CS-Faculty Office, 2nd Floor New Building
 Email: Adeel.munawar@lgu.edu.pk
 Visiting Hours: Friday 8am-10:30am. Also after class. Students can also walk in if
I am free
Friday, June
24, 2022
Lahore Garrison University
Analysis of Algorithm
2

Previous Lecture Summary
 Sorting in Linear Time
 Counting Sort
 Counting Sort Algorithm
 Radix Sort
Friday, June
24, 2022
3

Today’s Lecture Summary
 Dynamic Programming
 Fibonacci numbers
 Chain Matrix Multiplication
 Algorithm
 Example
Friday, June
24, 2022
4

Dynamic Programming
LOGICAL RE-USE OF COMPUTATIONS
5

Dynamic Programming
• Dynamic Programming is a general algorithm design
technique.
• Invented by American mathematician Richard Bellman in the
1950s to solve optimization problems.
• “Programming” here means “planning”.
• Main idea:
• solve several smaller (overlapping) subproblems.
• record solutions in a table so that each subproblem is
only solved once.
• final state of the table will be (or contain) solution. Friday, June
24, 2022
6

What is Dynamic Programming?
 Dynamic programming solves optimization problems by combining solutions to
subproblems
 “Programming” refers to a tabular method with a series of choices, not
“coding”
Friday, June
24, 2022
7

…contd
 Recall the divide-and-conquer approach(Top-down strategy)
 Partition the problem into independent subproblems
 Solve the subproblems recursively
 Combine solutions of subproblems
 This contrasts with the dynamic programming approach(bottom-up)
Friday, June
24, 2022
8

Friday, June
24, 2022
9
Top down Bottom Up
Follows the memorization technique Follows the tabulation method
Top Down Approach:
Memorization is equal to the sum of the recursion and caching whereas the
recursion is calling the function itself and caching is storing the intermediate
results.

Top Down Approach
Advantages
 It is very easy to understand and implement.
 It solves the subproblems only when it is required.
 It is easy to debug.
Disadvantages
 It uses the recursion technique that occupies more memory in the call stack.
Sometimes when the recursion is too deep, the stack overflow condition will
occur.
 It occupies more memory that degrades the overall performance.
Friday, June
24, 2022
10

Example
int fib(int n)
{
if(n<0)
error;
if(n==0)
return 0;
if(n==1)
return 1;
sum = fib(n-1) + fib(n-2);
}
Friday, June
24, 2022
11

Cont..
 In the above code, we have used the recursive approach to find out the Fibonacci
series. When the value of 'n' increases, the function calls will also increase, and
computations will also increase. In this case, the time complexity increases
exponentially, and it becomes 2n.
 One solution to this problem is to use the dynamic programming approach. Rather
than generating the recursive tree again and again, we can reuse the previously
calculated value. If we use the dynamic programming approach, then the time
complexity would be O(n).
Friday, June
24, 2022
12

…contd
 Dynamic programming is applicable when subproblems are not independent
 i.e., subproblems share subsubproblems
 Solve every subsubproblem only once and store the answer for use when it
reappears
 Bottom-up strategy
 A divide-and-conquer approach will do more work than necessary
Friday, June
24, 2022
13

 Divide-and-Conquer: a top-down approach.
 Many smaller instances are computed more than once.
 Dynamic programming: a bottom-up approach.
 Solutions for smaller instances are stored in a table for later use.
Why Dynamic Programming?
Friday, June
24, 2022
14

Why Dynamic Programming? …
The underlying idea of dynamic programming is thus quite simple:
avoid calculating the same thing twice, usually by keeping a table of
known results, which we fill up as subinstances are solved.
• Dynamic programming is a bottom-up technique.
• Examples:
1) Fibonacci numbers
2) Computing a Binomial coefficient
Friday, June
24, 2022
15

Dynamic Programming
 Fibonacci numbers:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34
 Recurrence Relation of Fibonacci numbers
?
Friday, June
24, 2022
16

Friday, June
24, 2022
17

Example: Fibonacci numbers
• Recall definition of Fibonacci numbers:
f(0) = 0
f(1) = 1
f(n) = f(n-1) + f(n-2) for n ≥ 2
• Computing the nth Fibonacci number recursively (top-
down): f(n)
f(n-1) + f(n-2)
f(n-2) + f(n-3) f(n-3) + f(n-4)
Friday, June
24, 2022
18

Recursive calls for fib(5)
fib(5)
fib(4) fib(3)
fib(3) fib(2) fib(2) fib(1)
fib(2) fib(1) fib(1) fib(0) fib(1) fib(0)
fib(1) fib(0)
Friday, June
24, 2022
19

fib(5) Using Dynamic Programming
fib(5)
fib(4) fib(3)
fib(3) fib(2) fib(2) fib(1)
fib(2) fib(1) fib(1) fib(0) fib(1) fib(0)
fib(1) fib(0)
6
5
2
1
3
4
Friday, June
24, 2022
20

Examples of Dynamic Programming Algorithms
• Computing binomial coefficients
• Optimal chain matrix multiplication
• Floyd’s algorithms for all-pairs shortest paths
• Constructing an optimal binary search tree
• Some instances of difficult discrete optimization problems:
• travelling salesman
• knapsack
Friday, June
24, 2022
21

A Sequence of 4 Steps
 A dynamic programming approach consists of a sequence of 4 steps
1. Characterize the structure of an optimal solution
2. Recursively define the value of an optimal solution
3. Compute the value of an optimal solution in a bottom-up fashion
4. Construct an optimal solution from computed information
Friday, June
24, 2022
22

MATRIX MULTIPLICATION
MATRIX MULTIPLY(A, B)
1. if columns[A]rows[B]
2. then error “incompatible dimension”
3. else
4. for i1 to rows[A]
5. do for j1 to columns[B]
6. do C[i,j] 0
7. for k1 to columns[A]
8. do C[i,j] C[i,j]+A[i,k]. B[k,j]
9. return C
Friday, June
24, 2022

MATRIX MULTIPLICATION
• Two matrices can only be multiplied if
columns[A] = rows [B]
• If order of A is p x q & order of B is q x r
then order of C will be p x r
• Time required to computer the product of two
matrices is p x q x r
Friday, June
24, 2022

MATRIX CHAIN MULTIPLICATION
Suppose we want to multiply a chain of matrices
A1A2A3
Let A1=10x20 A2=20x50 A3=50x20
We can do this in two different ways
(A1)(A2A3)=20*50*20+10*20*20=20000+4000=24000
(A1A2)(A3)=10*20*50+10*50*20=10000+10000=20000
Clearly the last option is optimal
We have to find the optimal way to parenthesize
the given sequence Friday, June
24, 2022

Friday, June
24, 2022
Matrix-Chain multiplication
Dynamic Programming – Bottom Up Approach
Example:
matrix dimension
A1 30 x 35
A2 35 x 15
A3 15 x 5
A4 5 x 10
A5 10 x 20
A6 20 x 25
Order Matrix P is:
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P

Friday, June
24, 2022
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P

August
2014
Amjad Ali
28
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
for l←2 to n
do for i←1 to n-l+1
do j←i+l-1
m[1,2]
m[2,3]
m[3,4]
m[4,5]
m[5,6]
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
l = 2
Dynamic Programming

29
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
for l←2 to n
do j←i+l-1
m[i, j] = min { m[i, k] +m[k+1, j] + pi-1 pk pj}
i ≤ k < j
m[1, 2] = min { m[1, 1] +m[2, 2] + p0 p1 p2}
1 ≤ k < 2
m[1, 2] = min { 0 + 0 + 30x35x15}
= min ( 15750 ) = 15750
m[2, 3] = min { m[2, 2] + m[3, 3] + p1 p2 p3}
2 ≤ k < 3
m[2, 3] = min { 0 + 0 + 35x15x5}
= min ( 2625 ) = 2625
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
l = 2
Dynamic Programming

30
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
for l←2 to n
do j←i+l-1
m[3, 4] = min { m[3, 3] +m[4, 4] + p2 p3 p4}
3 ≤ k < 4
m[3, 4] = min { 0 + 0 + 15x5x10}
= min ( 750 ) = 750
m[4, 5] = min { m[4, 4] + m[5, 5] + p3 p4 p5}
4 ≤ k < 5
m[4, 5] = min { 0 + 0 + 5x10x20}
= min ( 1000 ) = 1000
m[5, 6] = min { m[5, 5] + m[6, 6] + p4 p5 p6}
5 ≤ k < 6
m[5, 6] = min { 0 + 0 + 10x20x25}
= min ( 5000 ) = 5000
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
l = 2
Dynamic Programming

August
2014
Amjad Ali
31
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
for l←2 to n
do j←i+l-1
m[1, 3]
m[2, 4]
m[3, 5]
m[4, 6]
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
l = 3
Dynamic Programming

August
2014
Amjad Ali
32
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
for l←2 to n
do j←i+l-1
m[1, 3]=min { m[1, 1] +m[2, 3] + p0 p1 p3 ,
1 ≤ k < 3 m[1, 2] +m[3, 3] + p0 p2 p3 }
m[1, 3] = min { 0 + 2625 + 30x35x5 ,
15750 + 0 +30x15x5 }
= min ( 7875 , 18000 ) = 7875
m[2, 4] = min { m[2, 2] + m[3, 4] + p1 p2 p4 ,
2 ≤ k < 4 m [2, 3]+m[4, 4] + p1 p3 p4 }
m[2, 4] = min { 0 + 750 + 35x15x10 ,
2625 + 0 + 35x5x10}
= min ( 6000, 4375) = 4375
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
l = 3
Dynamic Programming

August
2014
Amjad Ali
33
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
for l←2 to n
do j←i+l-1
m[3, 5]=min { m[3, 3] +m[4, 5] + p2 p3 p5 ,
3 ≤ k < 5 m[3, 4] +m[5, 5] + p2 p4 p5 }
m[3, 5] = min { 0 + 1000 + 15x5x20 ,
750 + 0 +15x10x20 }
= min ( 2500 , 3750 ) = 2500
m[4, 6] = min { m[4, 4] + m[5, 6] + p3 p4 p6 ,
4 ≤ k < 6 m [4, 5]+m[6, 6] + p3 p5 p6 }
m[4, 6] = min { 0 + 5000 + 5x10x25 ,
1000 + 0 + 5x20x25}
= min ( 6250, 3500) = 3500
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
l = 3
Dynamic Programming

August
2014
Amjad Ali
34
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[1, 4]
m[2, 5]
m[3, 6]
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
l = 4
Dynamic Programming

August
2014
Amjad Ali
35
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[1, 4]=min { m[1, 1] + m[2, 4] + p0 p1 p4 ,
1 ≤ k < 4 m[1, 2] + m[3, 4] + p0 p2 p4 ,
m[1,3] + m[4,4] + p0 p3 p4 }
m[1, 4]=min { 0 + 4375 + 30x35x10 ,
15750 + 750 + 30x15x10
7875 + 0 + 30x5x10 }
m[1, 4]=min { 14875 , 21000 , 9375} = 9375
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
3
l = 4
Dynamic Programming

August
2014
Amjad Ali
36
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[2, 5]=min { m[2, 2] + m[3, 5] + p1 p2 p5 ,
2 ≤ k < 5 m[2, 3] + m[4, 5] + p1 p3 p5 ,
m[2,4] + m[5,5] + p1 p4 p5 }
m[2, 5]=min { 0 + 2500 + 35x15x20 ,
2625+ 1000 + 35x5x20 ,
4375 + 0 + 35x10x20}
m[2, 5]=min { 13000 , 7125 , 11375} = 7125
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
7125
3
3
l = 4
Dynamic Programming

August
2014
Amjad Ali
37
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[3, 6]=min { m[3, 3] + m[4, 6] + p2 p3 p6 ,
3 ≤ k < 6 m[3, 4] + m[5, 6] + p2 p4 p6 ,
m[3,5] + m[6,6] + p2 p5 p6 }
m[3, 6]=min { 0 + 3500 + 15x5x25 ,
750+ 5000 + 15x10x25 ,
2500 + 0 + 15x20x25}
m[3, 6]=min { 5375 , 9500 , 10000} = 5375
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
5375
7125
3
3
3
l = 4
Dynamic Programming

August
2014
Amjad Ali
38
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[1, 5]
m[2, 6]
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
5375
7175
3
3
3
l = 5
Dynamic Programming

August
2014
Amjad Ali
39
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[1, 5]=min { m[1, 1] + m[2, 5] + p0 p1 p5 ,
1 ≤ k < 5 m[1, 2] + m[3, 5] + p0 p2 p5 ,
m[1,3] + m[4,5] + p0 p3 p5 ,
m[1, 4] + m[5, 5] + p0 p4 p5 }
m[1, 5]=min { 0 + 7175 + 30x35x20 ,
15750 + 2500 + 30x15x20 ,
7875 + 1000 + 30x5x20 ,
9375 + 0 + 30x10x20}
m[1, 5]=min { 28175 , 27250 , 11875 , 15375}
= 11875
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
5375
7175
3
3
3
l = 5
11875
3
Dynamic Programming

August
2014
Amjad Ali
40
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[2, 6]=min { m[2, 2] + m[3, 6] + p1 p2 p6 ,
2 ≤ k < 6 m[2, 3] + m[4, 6] + p1 p3 p6 ,
m[2,4] + m[5,6] + p1 p4 p6 ,
m[2, 5] + m[6, 6] + p1 p5 p6 }
m[2, 6]=min { 0 + 5375 + 35x15x25 ,
2625+ 3500 + 35x5x25 ,
4375 + 5000 + 35x10x25 ,
5725 + 0 + 35x20x25}
m[2, 6]=min { 18500 , 10500 , 18125 , 23225}
= 10500
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
5375
5725
3
3
3
l = 5
3
10500
3
11875
Dynamic Programming

August
2014
Amjad Ali
41
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[1, 6]
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
5375
5725
3
3
3
l = 6
3
10500
3
11875
Dynamic Programming

August
2014
Amjad Ali
42
30 35 15 5 10 20 25
0 1 2 3 4 5 6
P
m[1, 6]=min { m[1, 1] + m[2, 6] + p0 p1 p6 ,
1 ≤ k < 6 m[1, 2] + m[3, 6] + p0 p2 p6 ,
m[1,3] + m[4,6] + p0 p3 p6 ,
m[1, 4] + m[5, 6] + p0 p4 p6 ,
m[1, 5] + m[6, 6] + p0 p5 p6 }
m[1, 6]=min { 0 + 10500 + 30x35x25,
15750+ 5375 + 30x15x25 ,
7875 + 3500 + 30x5x25 ,
9375+ 5000 + 30x10x25 ,
11875+ 0 + 30x20x25}
m[1, 6]=min { 36750 , 32375 , 15125 , 26875}
= 15125
1
2
3
4
6
5
1 2 3 4 5 6
1
2
3
4
6
5
1 2 3 4 5 6
m
S
0
0
0
0
0
0
15750
2625
1
2
750
3
1000
4
5000
5
7875
1
4375
3
2500
3
3500
5
9375
5375
5725
3
3
3
l = 6
3
10500
3
11875
3
15125
Dynamic Programming

3
3
3
3 3
3 3
3
1
5
2
1 3 4 5
s
15750
0
9375
7875
15125
11875
4375
2625
10500
7125
0
0
0
575
0
5000
3500
0
750
A1 A2 A3 A4 A5 A6
m
1000
2500
August
2014
Amjad Ali
43
Dynamic Programming

August
2014
Amjad Ali
44
Dynamic Programming

Summary
 Dynamic Programming
 Fibonacci numbers
 Chain Matrix Multiplication
 Algorithm
 Example
Friday, June
24, 2022
45

Q & A
Friday, June
24, 2022
46

References
 These lecture notes were taken from following source:
 Introduction to Algorithms 2nd ,Cormen, Leiserson, Rivest and
Stein, The MIT Press, 2001.
Friday, June
24, 2022
47

week 9 lec 15.pptx

More Related Content

Similar to week 9 lec 15.pptx

Recently uploaded

week 9 lec 15.pptx