Unit V.pdf

Unit-V Algorithm Design Techniques 9
Greedy algorithms –Scheduling problem-The multiprocessor case-Divide and conquer-Running
time –The Selection Problem- Dynamic programming – Recursive Algorithms-Ordering Matrix
Multiplication
Greedy algorithms:
 "Greedy Method finds out of many options, but you have to choose the best option."
 In this method, we have to find out the best method/option out of many present ways.
 In this approach/method we focus on the first stage and decide the output, don't think
about the future.
 This method may or may not give the best output.
Greedy Algorithm solves problems by making the best choice that seems best at the particular
moment. Many optimization problems can be determined using a greedy algorithm. Some issues
have no efficient solution, but a greedy algorithm may provide a solution that is close to optimal.
A greedy algorithm works if a problem exhibits the following two properties:
1. Greedy Choice Property: A globally optimal solution can be reached at by creating a
locally optimal solution. In other words, an optimal solution can be obtained by creating
"greedy" choices.
2. Optimal substructure: Optimal solutions contain optimal subsolutions. In other words,
answers to subproblems of an optimal solution are optimal.
Example:
1. machine scheduling
2. Fractional Knapsack Problem
3. Minimum Spanning Tree
4. Huffman Code
5. Job Sequencing
6. Activity Selection Problem

Steps for achieving a Greedy Algorithm are:
1. Feasible: Here we check whether it satisfies all possible constraints or not, to obtain at
least one solution to our problems.
2. Local Optimal Choice: In this, the choice should be the optimum which is selected from
the currently available
3. Unalterable: Once the decision is made, at any subsequence step that option is not
altered.
Applications of Greedy Algorithm
1. Fit algorithm in memory management
Fit memory management algorithms are techniques for allocating the memory blocks to the
processes in an operating system.
Here, a pointer keeps track of all the memory blocks and accepts or rejects the request of
allocating a memory block to an incoming process. There are three types of fit memory
management algorithms:
1. First fit algorithm: First sufficient memory block from the top of the main memory is
allocated to the process.
2. Best fit algorithm: Minimum sufficient memory block available in the memory is allocated
to the process.
3. Worst fit algorithm: Largest sufficient memory block available in the memory is allocated
to the process.
2. Scheduling algorithms
A typical process in a computer has input time, output time, and CPU time. In multiprogramming
systems, while one process is waiting for its input/output the other processes can use the CPU.
Process scheduling algorithms perform these tasks. These algorithms determine which process
will use the CPU for execution and which processes will be on hold. These algorithms make sure
that the CPU utilization is for maximum time and process execution takes place in minimum
time.
Six types of scheduling algorithms are:
a. First come first serve: In this non-preemptive scheduling algorithm, the processes are
executed on a first come first serve basis.
b. Shortest job first: In this non-preemptive scheduling algorithm, the approach is to minimize
the waiting time for the process.
c. Priority scheduling: This non-preemptive algorithm assigns each process a priority, and the
process with the highest priority is executed first.

d. Shortest remaining time: In this preemptive scheduling algorithm, the process which has the
least time remaining for its completion is executed first.
e. Round robin scheduling: In this preemptive process scheduling algorithm, each process is
provided a fixed time to get executed. If the process is not completely executed, it will be
assigned the same time for execution in the next round.
f. Multiple level queue schedule: These are not independent scheduling algorithms.
Programmers make many queues for processes with common characteristics and every queue can
have its own scheduling algorithm.
3. Bin packing problem
We need to pack items of different volumes into a finite number of bins of fixed volume each in
such a way that we use the minimum number of bins. Placing data on multiple disks and loading
containers like trucks are some real-life examples of this algorithm.
4. Travelling salesman problem
Given a set of cities and distance between every pair of the city, the objective is to find the
shortest possible path to visit each city exactly once and return to the starting point.
5. Fractional knapsack problem
Imagine you have different items with their weights and their values. Our objective is to
determine the subset of items that we should include in a collection so that their total weight is
less than or equal to a given limit and their value is maximum. The fractional knapsack problem
is an optimization problem.
6. Minimum spanning trees
Minimum spanning tree is a subset of the edges from starting to end in a weighted and
undirected graph. MST connects all vertices without any cycle and the total weight of all the
edges included is minimum..
7. Dijkstra shortest path algorithm
Finding the shortest path between two vertices in a graph such that the sum of weights between
the edges is minimum.
8. Egyptian fraction
We can represent every positive fraction as a sum of unique unit fractions. For example, the
Egyptian fraction representation of 47/154 is 1/7 + 1/11 + 1/14.

Scheduling problem
The activity search problem is a mathematical search optimization problem. This is used in
scheduling a single resource among several ongoing and incoming processes. A greedy
algorithm provides the best solution for selecting the maximum size set of compatible activities.
Let S = {1,2,3,…n) be a set of n proposed processes. Resources can be used by only one process
at a time. Start time of activities = {s1,s2,s3,….sn} and finish time = {f1,f2,f3,…fn}. Start time
is always less than finish time.
If activity “i” starts meanwhile the half-open time interval [si, fi), then, activities i and j are said
to be compatible if the intervals (si, fi) and [si, fi) do not overlap.
Algorithm:
activity_selector (s, f)
Step 1: n ← len[s]
Step 2: A ← {1}
Step 3: j ← 1.
Step 4: for i ← 2 to n
Step 5: do if si ≥ fi
Step 6: then A ← A U {i}
Step 7: j ← i
Step 8: return A
Example :
S={A1,A2,A3,A4,A5}
si={1,1,2,3,4}
fi={4,2,4,7,6}
Step 1. Arrange in increasing order of finish time.
Activity A2 A3 A1 A5 A4
si 1 2 1 4 3
fi 2 4 4 6 7
Step 2: Schedule A2

Step 3: Schedule A3 (A2, A3 not interfering)
Step 4: Skip A1 (Interfering)
Step 5: Schedule A5 (A2, A3, A5 not interfering)
Step 6: Skip A4 (Interfering)
Step 7:Final Activity schedule is A2, A3, A5.
Divide and conquer
In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and
then each problem is solved independently. When we keep on dividing the subproblems into
even smaller sub-problems, we may eventually reach a stage where no more division is
possible. Those "atomic" smallest possible sub-problem (fractions) are solved. The solution of
all sub-problems is finally merged in order to obtain the solution of an original problem.
Broadly, we can understand divide-and-conquer approach in a three-step process.

Divide/Break
This step involves breaking the problem into smaller sub-problems. Sub-problems should
represent a part of the original problem. This step generally takes a recursive approach to divide
the problem until no sub-problem is further divisible. At this stage, sub-problems become
atomic in nature but still represent some part of the actual problem.
Conquer/Solve
This step receives a lot of smaller sub-problems to be solved. Generally, at this level, the
problems are considered 'solved' on their own.
Merge/Combine
When the smaller sub-problems are solved, this stage recursively combines them until they
formulate a solution of the original problem. This algorithmic approach works recursively and
conquer & merge steps works so close that they appear as one.
Examples
The following computer algorithms are based on divide-and-conquer programming approach −
 Merge Sort
 Quick Sort
 Binary Search
 Strassen's Matrix Multiplication
 Closest pair (points)

There are various ways available to solve any computer problem, but the mentioned are a good
example of divide and conquer approach.
Fundamental of Divide & Conquer Strategy:
There are two fundamental of Divide & Conquer Strategy:
1. Relational Formula
2. Stopping Condition
1. Relational Formula: It is the formula that we generate from the given technique. After
generation of Formula we apply D&C Strategy, i.e. we break the problem recursively & solve
the broken subproblems.
2. Stopping Condition: When we break the problem using Divide & Conquer Strategy, then we
need to know that for how much time, we need to apply divide & Conquer. So the condition
where the need to stop our recursion steps of D&C is called as Stopping Condition.
Applications of Divide and Conquer Approach:
Following algorithms are based on the concept of the Divide and Conquer Technique:
1. Binary Search: The binary search algorithm is a searching algorithm, which is also called
a half-interval search or logarithmic search. It works by comparing the target value with
the middle element existing in a sorted array. After making the comparison, if the value
differs, then the half that cannot contain the target will eventually eliminate, followed by
continuing the search on the other half. We will again consider the middle element and
compare it with the target value. The process keeps on repeating until the target value is
met. If we found the other half to be empty after ending the search, then it can be
concluded that the target is not present in the array.
2. Quicksort: It is the most efficient sorting algorithm, which is also known as partition-
exchange sort. It starts by selecting a pivot value from an array followed by dividing the
rest of the array elements into two sub-arrays. The partition is made by comparing each
of the elements with the pivot value. It compares whether the element holds a greater
value or lesser value than the pivot and then sort the arrays recursively.
3. Merge Sort: It is a sorting algorithm that sorts an array by making comparisons. It starts
by dividing an array into sub-array and then recursively sorts each of them. After the
sorting is done, it merges them back.

4. Closest Pair of Points: It is a problem of computational geometry. This algorithm
emphasizes finding out the closest pair of points in a metric space, given n points, such
that the distance between the pair of points should be minimal.
5. Strassen's Algorithm: It is an algorithm for matrix multiplication, which is named after
Volker Strassen. It has proven to be much faster than the traditional algorithm when
works on large matrices.
6. Cooley-Tukey Fast Fourier Transform (FFT) algorithm: The Fast Fourier Transform
algorithm is named after J. W. Cooley and John Turkey. It follows the Divide and
Conquer Approach and imposes a complexity of O(nlogn).
7. Karatsuba algorithm for fast multiplication: It is one of the fastest multiplication
algorithms of the traditional time, invented by Anatoly Karatsuba in late 1960 and got
published in 1962. It multiplies two n-digit numbers in such a way by reducing it to at
most single-digit.
Advantages of Divide and Conquer
o Divide and Conquer tend to successfully solve one of the biggest problems, such as the
Tower of Hanoi, a mathematical puzzle. It is challenging to solve complicated problems
for which you have no basic idea, but with the help of the divide and conquer approach, it
has lessened the effort as it works on dividing the main problem into two halves and then
solve them recursively. This algorithm is much faster than other algorithms.
o It efficiently uses cache memory without occupying much space because it solves simple
subproblems within the cache memory instead of accessing the slower main memory.
o It is more proficient than that of its counterpart Brute Force technique.
o Since these algorithms inhibit parallelism, it does not involve any modification and is
handled by systems incorporating parallel processing.
Disadvantages of Divide and Conquer
o Since most of its algorithms are designed by incorporating recursion, so it necessitates
high memory management.
o An explicit stack may overuse the space.
o It may even crash the system if the recursion is performed rigorously greater than the
stack present in the CPU.

Running time of Algorithms
 The running time of an algorithm for a specific input depends on the number of
operations executed.
 The greater the number of operations, the longer the running time of an algorithm.
Big-O
Big-O is the shorthand used to classify the time complexity of algorithms. It has a formal
mathematical definition, but you just need to know how to classify algorithms into different
"Big-O categories", which can be reviewed below:
O(1) - Constant Time
The algorithm performs a constant number of operations regardless of the size of the input.
Examples:
 Access a single number in an array by index
 Add 2 numbers together.
O(log n) - Logarithmic Time
As the size of input n increases, the algorithm's running time grows by log(n). This rate of
growth is relatively slow, so O(log n) algorithms are usually very fast. As you can see in the
table below, when n is 1 billion, log(n) is only 30.
Example:
Binary search on a sorted list. The size that needs to be searched is split in half each time, so the
remaining list goes from size n to n/2 to n/4... until either the element is found or there's just one
element left. If there are a billion elements in the list, it will take a maximum of 30 checks to find
the element or determine that it is not on the list.
Whenever an algorithm only goes through part of its input, see if it splits the input by at least half
on average each time, which would give it a logarithmic running time.
Table of n vs. log(n)
n log(n)
2 1

1000 10
1M 20
1B 30
O(n) - Linear Time
The running time of an algorithm grows in proportion to the size of input.
Examples:
 Search through an unsorted array for a number
 Sum all the numbers in an array
 Access a single number in a LinkedList by index
O(n log n) - Linearithmic time
log(n) is much closer to n than to n2
, so this running time is closer to linear than to higher running
times (and no one actually says "Linearithmic").
Examples:
Sort an an array with QuickSort or Merge Sort. When recursion is involved, calculating the
running time can be complicated. You can often work out a sample case to estimate what the
running time will be. See Quick Sort for more info.
O(n2)
- Quadratic Time
The algorithm's running time grows in proportion to the square of the input size, and is common
when using nested-loops.
Examples:
 Printing the multiplication table for a list of numbers
 Insertion Sort
Table of n vs. n2
n n2

1 1
10 100
100 10,000
1000 1M
Nested Loops
Sometimes you'll encounter O(n3)
algorithms or even higher exponents. These usually have
considerably worse running times than O(n2)
algorithms even if they don't have a different name!
A Quadratic running time is common when you have nested loops. To calculate the running
time, find the maximum number of nested loops that go through a significant portion of the
input.
 1 loop (not nested) = O(n)
 2 loops = O(n2)
 3 loops = O(n3)
Some algorithms use nested loops where the outer loop goes through an input n while the inner
loop goes through a different input m. The time complexity in such cases is O(nm). For example,
while the above multiplication table compared a list of data to itself, in other cases you may want
to compare all of one list n with all of another list m.
O(2n)
- Exponential Time
Table of n vs. 2n
(reverse of log(n) table)
n 2n
1 2
10 ~1000
20 ~1M

30 ~1B
Example:
 Trying out every possible binary string of length n. (E.g. to crack a password).
O(n!) - Factorial Time
This algorithm's running time grows in proportion to n!, a really large number. O(n!) becomes
too large for modern computers as soon as n is greater than 15+ (or upper teens). This issue
shows up when you try to solve a problem by trying out every possibility, as in the traveling
salesman problem.
Examples:
 Go through all Permutations of a string
 Traveling Salesman Problem (brute-force solution)
Table of n vs. n!
n n!
1 1
10 ~3.6M
20 ~2.43 × 1018
30 ~2.65 × 1032
Basic operations
Knowing the cost of basic operations helps to calculate the overall running time of an algorithm.
The table below shows the list of basic operations along with their running time. The list not by
any means provides the comprehensive list of all the operations. But I am trying to include most
of the operations that we come across frequently in programming.
Operation Running Time

Integer add/subtract Θ(1)Θ(1)
Integer multiply/divide Θ(1)Θ(1)
Float add/subtract Θ(1)Θ(1)
Float multiply/divide Θ(1)Θ(1)
Trigonometric Functions (sine, cosine, ..) Θ(1)Θ(1)
Variable Declaration Θ(1)Θ(1)
Assignment Operation Θ(1)Θ(1)
Logical Operations (<,>,≤,≥,<,>,≤,≥, etc) Θ(1)Θ(1)
Array Access Θ(1)Θ(1)
Array Length Θ(1)Θ(1)
1D array allocation Θ(n)Θ(n)
2D array allocation Θ(n2)Θ(n2)
Substring extraction Θ(1)Θ(1) or Θ(n)Θ(n)
String concatenation Θ(n)Θ(n)
Consecutive statements
Let two independent consecutive statements are P1P1 and P2P2. Let t1t1 be the cost of
running P1P1 and t2t2 be the cost of running P2P2. The total cost of the program is the addition

of cost of individual statement i.e. t1+t2t1+t2. In asymptotic notation the total time
is Θ(max(t1,t2))Θ(max(t1,t2))(we ignore the non significant term).
Example: Consider the following code.
1
2
3
4
5
int main() {
// 1. some code with running time n
// 2. some code with running time n^2
return 0;
}
Assume that statement 2 is independent of statement 1 and statement 1 executes first followed by
statement 2. The total running time is
Θ(max(n,n2))=Θ(n2)Θ(max(n,n2))=Θ(n2)
for loops
It is relatively easier to compute the running time of for loop than any other loops. All we need to
compute the running time is how many times the statement inside the loop body is executed.
Consider a simple for loop in C.
1
2
3
for (i = 0; i < 10; i++) {
// body
}
The loop body is executed 10 times. If it takes mm operations to run the body, the total number of
operations is 10×m=10m10×m=10m. In general, if the loop iterates nn times and the running time
of the loop body are mm, the total cost of the program is n mn m. Please note that we are
ignoring the time taken by expression i<10i<10 and statement i++i++. If we include these, the
total time becomes
1+2×n+mn=Θ(mn)1+2×n+mn=Θ(mn)
In this analysis, we made one important assumption. We assumed that the body of the loop
doesn’t depend on i. Sometimes the runtime of the body does depend on i. In that case, our
calculation becomes a little bit difficult. Consider an example shown below.
1
2
3
4
5
for (i = 0; i < n; i++) {
if (i % 2 == 0) {
// some operations of runtime n^2
}

}
In the for loop above, the control goes inside the if condition only when i is an even number. That
means the body of if condition gets executed n/2n/2 times. The total cost is
therefore n/2 n2=n3/2=Θ(n3)n/2 n2=n3/2=Θ(n3).
Nested for loops
Suppose there are pp nested for loops. The pp for loops execute n1,n2,…,npn1,n2,…,np times
respectively. The total cost of the entire program is
n1×n2×,….,×np×cost of the body of innermost loopn1×n2×,….,×np×cost of the body of
innermost loop
Consider nested for loops as given in the code below
1
2
3
4
5
for (i = 0; i < n; i++) {
for (j = 0; j < n; j++) {
// body that runs in linear time n
}
}
There are two for loops, each goes n times. So the total cost is
n×n×n=n3=Θ(n3)n×n×n=n3=Θ(n3)
while loops
while loops are usually harder to analyze than for loops because there is no obvious a priori way
to know how many times we shall have to go round the loop. One way of analyzing while loops is
to find a variable that goes increasing or decreasing until the terminating condition is met.
Consider an example given below
1
2
3
4
while (i > 0) {
// some computation of cost n
i = i / 2
}
How many times the loop repeats? In every iteration, the value of i gets halved. If the initial value
of i is 16, after 4 iterations it becomes 1 and the loop terminates. The implies that the loop
repeats log2ilog2 i times. In each iteration, it does the nn work. Therefore the total cost
is Θ(nlog2i)Θ(nlog2 i).

Recursive calls
To calculate the cost of a recursive call, we first transform the recursive function to a recurrence
relation and then solve the recurrence relation to get the complexity. There are many techniques to
solve the recurrence relation. These techniques will be discussed in details in the next article.
1
2
3
4
5
6
7
int fact(int n) {
if (n <= 2) {
return n;
}
return n * fact(n - 1);
}
We can transform the code into a recurrence relation as follows.
T(n)={ab+T(n−1)if n≤2otherwiseT(n)={aif n≤2b+T(n−1)otherwise
When n is 1 or 2, the factorial of n is nn itself. We return the result in constant time aa.
Otherwise, we calculate the factorial of n−1n−1 and multiply the result by nn. The multiplication
takes a constant time bb. We use one of the techniques called back substitution to find the
complexity.
T(n)=b+T(n−1)=b+b+T(n−2)=b+b+b+T(n−3)=3b+T(n−3)=kb+T(n−k)=nb+T(0)=nb+a=Θ(n)T(n
)=b+T(n−1)=b+b+T(n−2)=b+b+b+T(n−3)=3b+T(n−3)=kb+T(n−k)=nb+T(0)=nb+a=Θ(n)
Example
Let us put together all the techniques discussed above and compute the running time of some
example programs.
Example 1
1
2
3
4
int sum(int a, int b) {
int c = a + b;
return c
}
The sum function has two statements. The first statement (line 2) runs in constant time
i.e. Theta(1)Theta(1) and second statement (line 3) also runs in constant time Θ(1)Θ(1). These
two statements are consecutive statements, so the total running time
is Θ(1)+Θ(1)=Θ(1)Θ(1)+Θ(1)=Θ(1)

Example 2
1
2
3
4
5
6
7
8
int array_sum(int a, int n) {
int i;
int sum = 0;
for (i = 0; i < n; i++) {
sum = sum + a[i]
}
return sum;
}
Analysis
1. Line 2 is a variable declaration. The cost is Θ(1)Θ(1)
2. Line 3 is a variable declaration and assignment. The cost is Θ(2)Θ(2)
3. Line 4 - 6 is a for loop that repeats nn times. The body of the for loop requires Θ(1)Θ(1) to
run. The total cost is Θ(n)Θ(n).
4. Line 7 is a return statement. The cost is Θ(1)Θ(1).
1, 2, 3, 4 are consecutive statements so the overall cost is Θ(n)Θ(n)
Example 3
1
2
3
4
5
6
7
8
9
10
11
12
int sum = 0;
for (i = 0; i < n; i++) {
for (j = 0; j < n; j++) {
for (k = 0; k < n; k++) {
if (i == j == k) {
for (l = 0; l < n*n*n; l++) {
sum = i + j + k + l;
}
}
}
}
}
Analysis
1. Line 1 is a variable declaration and initialization. The cost is Θ(1)Θ(1)
2. Line 2 - 11 is a nested for loops. There are four for loops that repeat nn times. After the
third for loop in Line 4, there is a condition of i == j == k. This condition is true
only nn times. So the total cost of these loops is Θ(n3)+Θ(n4)=Θ(n4)Θ(n3)+Θ(n4)=Θ(n4)

The overall cost is Θ(n4)Θ(n4).
Dynamic programming
Dynamic programming approach is similar to divide and conquer in breaking down the problem
into smaller and yet smaller possible sub-problems. But unlike, divide and conquer, these sub-
problems are not solved independently. Rather, results of these smaller sub-problems are
remembered and used for similar or overlapping sub-problems.
Dynamic programming is used where we have problems, which can be divided into similar sub-
problems, so that their results can be re-used. Mostly, these algorithms are used for
optimization. Before solving the in-hand sub-problem, dynamic algorithm will try to examine
the results of the previously solved sub-problems. The solutions of sub-problems are combined
in order to achieve the best solution.
So we can say that −
 The problem should be able to be divided into smaller overlapping sub-problem.
 An optimum solution can be achieved by using an optimum solution of smaller sub-
problems.
 Dynamic algorithms use Memoization.
Comparison
In contrast to greedy algorithms, where local optimization is addressed, dynamic algorithms are
motivated for an overall optimization of the problem.
In contrast to divide and conquer algorithms, where solutions are combined to achieve an
overall solution, dynamic algorithms use the output of a smaller sub-problem and then try to
optimize a bigger sub-problem. Dynamic algorithms use Memoization to remember the output
of already solved sub-problems.
Example
The following computer problems can be solved using dynamic programming approach −
 Fibonacci number series
 Knapsack problem
 Tower of Hanoi
 All pair shortest path by Floyd-Warshall
 Shortest path by Dijkstra
 Project scheduling
Dynamic programming can be used in both top-down and bottom-up manner. And of course,
most of the times, referring to the previous solution output is cheaper than recomputing in terms
of CPU cycles.

Approaches of dynamic programming
There are two approaches to dynamic programming:
Top-down approach
Bottom-up approach
Top-down approach
The top-down approach follows the memorization technique, while bottom-up approach
follows the tabulation method. Here memorization is equal to the sum of recursion and
caching. Recursion means calling the function itself, while caching means storing the
intermediate results.
Advantages
It is very easy to understand and implement.
It solves the subproblems only when it is required.
It is easy to debug.
Disadvantages
It uses the recursion technique that occupies more memory in the call stack. Sometimes when
the recursion is too deep, the stack overflow condition will occur.
It occupies more memory that degrades the overall performance.
Let's understand dynamic programming through an example.
int fib(int n)
{
if(n<0)
error;
if(n==0)
return 0;
if(n==1)
return 1;
sum = fib(n-1) + fib(n-2);

}
In the above code, we have used the recursive approach to find out the Fibonacci series.
When the value of 'n' increases, the function calls will also increase, and computations will
also increase. In this case, the time complexity increases exponentially, and it becomes 2n
.
One solution to this problem is to use the dynamic programming approach. Rather than
generating the recursive tree again and again, we can reuse the previously calculated value. If
we use the dynamic programming approach, then the time complexity would be O(n).
When we apply the dynamic programming approach in the implementation of the Fibonacci
series, then the code would look like:
static int count = 0;
int fib(int n)
{
if(memo[n]!= NULL)
return memo[n];
count++;
if(n<0)
error;
if(n==0)
return 0;
if(n==1)
return 1;
sum = fib(n-1) + fib(n-2);
memo[n] = sum;
}
In the above code, we have used the memorization technique in which we store the results in
an array to reuse the values. This is also known as a top-down approach in which we move
from the top and break the problem into sub-problems.
Bottom-Up approach
The bottom-up approach is also one of the techniques which can be used to implement the
dynamic programming. It uses the tabulation technique to implement the dynamic
programming approach. It solves the same kind of problems but it removes the recursion. If
we remove the recursion, there is no stack overflow issue and no overhead of the recursive

functions. In this tabulation technique, we solve the problems and store the results in a
matrix.
There are two ways of applying dynamic programming:
Top-Down
Bottom-Up
The bottom-up is the approach used to avoid the recursion, thus saving the memory space.
The bottom-up is an algorithm that starts from the beginning, whereas the recursive
algorithm starts from the end and works backward. In the bottom-up approach, we start from
the base case to find the answer for the end. As we know, the base cases in the Fibonacci
series are 0 and 1. Since the bottom approach starts from the base cases, so we will start from
0 and 1.
Key points
 We solve all the smaller sub-problems that will be needed to solve the larger sub-
problems then move to the larger problems using smaller sub-problems.
 We use for loop to iterate over the sub-problems.
 The bottom-up approach is also known as the tabulation or table filling method.
Example: Fractional Knapsack Problem
Optimization problem where it is possible to select fractional items rather than binary choices of
0 and 1. The binary problem cannot be solved by a greedy approach.
Step 1: Calculate the value per pound.
Step 2: Take the maximum possible amount of substance with the maximum value per pound.
Step 3: If that amount is exhausted, and we have space, take the maximum possible amount of
substance with the next maximum value per pound.
Step 4: Sort the item values per pound, greedy algorithm has a time complexity of O(n log n ).
Algorithm:
fractional knapsack (Array w, Array v, int C)
1. for i= 1 to size (v)
2. do vpp [i] = v [i] / w [i]
3. Sort-Descending (vpp)
4. i ← 1
5. while (C>0)
6. do amt = min (C, w [i])
7. sol [i] = amt
8. C= C-amt
9. i ← i+1

10. return sol
Example:
I = { I1,I2,I3,I4}
w={ 20,30,40,50}
v={40,20,70,60}
Capacity, C = 75
Step 1: vpp = { 2,0.66,1.75,1.2}
Step 2: Choose I1. Total weight 20. Left Capacity = 40
Step 3: Choose I3, Total weight 40. Left capacity = 15
Step 4: Choose I4 fractional. Total weight 15. Left capacity = 0
Step 5: value = 40+70+1.2*15 = 128
Recursive Algorithms
 A recursive algorithm calls itself which usually passes the return value as a parameter to
the algorithm again.
 This parameter is the input while the return value is the output.
 Recursive algorithm is a method of simplification that divides the problem into sub-
problems of the same nature.
 The result of one recursion is the input for the next recursion. The repletion is in the self-
similar fashion.
 Generation of factorial, Fibonacci number series are the examples of recursive
algorithms.
Properties:
 A recursive function can go infinite like a loop. To avoid infinite running of recursive
function, there are two properties that a recursive function must have −
 Base criteria − There must be at least one base criteria or condition, such that, when this
condition is met the function stops calling itself recursively.
 Progressive approach − The recursive calls should progress in such a way that each time
a recursive call is made it comes closer to the base criteria.
Types of Recursion
1. Primitive Recursion
2. Tail Recursion
3. Single Recursion
4. Multiple Recursion

5. Mutual Recursion or Indirect Recursion
6. General Recursion
1.Primitive Recursion
 It is the types of recursion that can be converted into a loop.
 We have already seen the Fibonacci series example which can be programmed with
recursion as well as with loop.
2. Tail Recursion
 It is a primitive recursion in which the recursive call is present as the last thing in the
function.
 In the above Fibonacci example, the recursive function is executed as the last statement
of the ‘fibo’ function.
3. Single Recursion
 It is the types of recursion when there is only one recursive call in the function.
4. Multiple Recursion
 As by its name, it is the types of recursion when there are multiple recursive calls in the
function.
 It is just the opposite of a single recursion. Recursion can be either single or multiple
type.
5. Mutual Recursion or Indirect Recursion)
 There are two or more functions involved in this type of recursion.
 In this type of recursion, a function calls another function, which eventually calls the
original function.
6. General Recursion
 If there is a function which cannot be defined without recursion, is called as general
recursion.
 It is the opposite of primitive type recursion.
Examples:
1. Towers of Hanoi (TOH)

2. Inorder/Preorder/Postorder Tree Traversals
3. DFS of Graph, etc.
Towers of Hanoi (TOH)
 Tower of Hanoi, is a mathematical puzzle which consists of three towers (pegs) and more
than one rings is as depicted −
 These rings are of different sizes and stacked upon in an ascending order, i.e. the smaller
one sits over the larger one. There are other variations of the puzzle where the number of
disks increase, but the tower count remains the same.
Rules
 The mission is to move all the disks to some another tower without violating the
sequence of arrangement. A few rules to be followed for Tower of Hanoi are −
 Only one disk can be moved among the towers at any given time.
 Only the "top" disk can be removed.
 No large disk can sit over a small disk.
Example
 Solve the Tower of Hanoi puzzle with three disks.
Solution:
 Tower of Hanoi puzzle with n disks can be solved in minimum 2^n−1 steps.
 This presentation shows that a puzzle with 3 disks has taken 2^3 - 1 = 7 steps.

A recursive algorithm for Tower of Hanoi
START
Procedure Hanoi(disk, source, dest, aux)
IF disk == 1, THEN
move disk from source to dest
ELSE
Hanoi(disk - 1, source, aux, dest) // Step 1
move disk from source to dest // Step 2
Hanoi(disk - 1, aux, dest, source) // Step 3
END IF
END Procedure
STOP
Ordering Matrix Multiplication

 Matrix chain multiplication (or the matrix chain ordering problem[citation needed]) is
an optimization problem concerning the most efficient way to multiply a given sequence
of matrices.
 The problem is not actually to perform the multiplications, but merely to decide the
sequence of the matrix multiplications involved.
 The problem may be solved using dynamic programming.
 There are many options because matrix multiplication is associative.
 In other words, no matter how the product is parenthesized, the result obtained will
remain the same.
 For example, for four matrices A, B, C, and D, there are five possible options:
((AB)C)D = (A(BC))D = (AB)(CD) = A((BC)D) = A(B(CD)).
 Although it does not affect the product, the order in which the terms are parenthesized
affects the number of simple arithmetic operations needed to compute the product, that is,
the computational complexity.
 For example, if A is a 10 × 30 matrix, B is a 30 × 5 matrix, and C is a 5 × 60 matrix, then
 computing (AB)C needs (10×30×5) + (10×5×60) = 1500 + 3000 = 4500 operations,
while computing A(BC) needs (30×5×60) + (10×30×60) = 9000 + 18000 = 27000
operations.
The multiprocessor case
 Multiprocessing, in computing, a mode of operation in which two or more processors in a
computer simultaneously process two or more different portions of the same program (set
of instructions).
 Multiprocessing is typically carried out by two or more microprocessors, each of which is
in effect a central processing unit (CPU) on a single tiny chip.
 Supercomputers typically combine millions of such microprocessors to interpret and
execute instructions.

Advantages of multiprocessing
 Increased reliability: Due to the multiprocessing system, processing tasks can be
distributed among several processors. This increases reliability as if one processor fails;
the task can be given to another processor for completion.
 Increased throughout: As several processors increase, more work can be done in less
 The economy of Scale: As multiprocessors systems share peripherals, secondary storage
devices, and power supplies, they are relatively cheaper than single-processor systems.
Disadvantages of multiprocessing
 multiprocessing is more complex and sophisticated as it takes care of multiple CPUs at
the same time.
Types of multiprocessing systems
 Symmetrical multiprocessing operating system
 Asymmetric multiprocessing operating system

Symmetrical multiprocessing
 In a Symmetrical multiprocessing system, each processor executes the same copy of the
operating system, takes its own decisions, and cooperates with other processes to smooth
the entire functioning of the system.
 The CPU scheduling policies are very simple. Any new job submitted by a user can be
assigned to any processor that is least burdened. It also results in a system in which all
processors are equally burdened at any time.
 The symmetric multiprocessing operating system is also known as a "shared every-thing"
system, because the processors share memory and the Input output bus or data path. In
this system processors do not usually exceed more than 16.
Characteristics of Symmetrical multiprocessing
 In this system, any processor can run any job or process.
 In this, any processor initiates an Input and Output operation.
Asymmetric multiprocessing
 In an asymmetric multiprocessing system, there is a master slave relationship between
the processors.
 Further, one processor may act as a master processor or supervisor processor while others
are treated as shown below.

Advantages of multiprocessing
 In this type of system execution of Input and Output operation or an application program
may be faster in some situations because many processors may be available for a single
job.
Disadvantages of multiprocessing
 In this type of multiprocessing operating system the processors are unequally burdened.
One processor may be having a long job queue, while another one may be sitting idle.
 In this system, if the process handling a specific work fails, the entire system will go
down.

Unit V.pdf

More Related Content

Similar to Unit V.pdf

Recently uploaded

Unit V.pdf