1.
Algorithms
1
BSc CSIT
Tribhuvan University Kathmandu
By:
Dabal Singh Mahara
2017

2.
Unit - 2
Contents Hours Marks
i. Concept and Definition
ii. Characteristics of Algorithm
iii. Design of Algorithm
iv. Big O Notation
2 3
2

3.
• The term algorithm is a corruption of the name 'al-Khowarizmi', a
mathematician of the ninth century Abu Jafar Muhammad Ibn Musa Al-
Khowarizmi.
• Originally, the term 'algorism' was used for the rules for performing
arithmetic using decimal notation.
• Algorism was evolved into the word algorithm by the eighteenth century.
• With the growing interests in computing machines, the concept of an
algorithm was given a more general meaning, to include all definite
procedures for solving problems, not just the procedures for performing
arithmetic.
Evolution of the term Algorithm
3

4.
Concept and Definition : Algorithm
• Informally, an algorithm is any well-defined computational procedure that takes
some value, or set of values, as input and produces some value, or set of values as
output.
• An algorithm is thus a sequence of computational steps that transform the input into
the output.
• We can view an algorithm as a tool for solving a well-specified computational
problem.
• The statement of the problem specifies in general terms the desired input/output
relationship.
• The algorithm describes a specific computational procedure for achieving that
input/output relationship.
• Therefore, an algorithm is a precise specification of a sequence of instructions to be
carried out in order to solve a given problem.
4

5.
Example
• One might need to sort a sequence of numbers into non-decreasing order.
• This problem arises frequently in practice and let us define this sorting
problem:
o Input: A sequence of n numbers { a1,a2, ...,an}.
o Output: A reordering {𝑎1
′
, 𝑎2
′
, ... , 𝑎 𝑛
′ } of the sequence of input sequence
such that 𝑎1
′
≤ 𝑎2
′
≤....≤𝑎 𝑛
′
o For example: given the input sequence {31,14,25,40,55,17}, a sorting
algorithm returns as output sequence {14,17,25,31,40,55}.
o Such an input sequence is called an instance of the sorting problem.
• An algorithm is said to be correct if, for every input instance, it halts with
the correct output.
5

6.
Problem, Algorithm and Program
• A problem can be viewed as a function that has a well-defined value or set of values
as input and a value of a set of values as expected output.
• Algorithm is implementation of this function that transforms input to corresponding
output.
• A problem can be solved by many different algorithms. But, a given algorithm solves
only one problem.
• For example: given a set of numbers, we have to sort these numbers. This problem
of sorting can be solved by many different sorting algorithms such as bubble sort,
insertion sort, selection sort, quick sort, merge sort, heap sort, bucket sort and so on.
• But each of these algorithm solves only problem of sorting nothing else.
• The advantage of knowing different solutions to the same problem is that we can
select the best solution for our problem.
• The program is an instance or concrete representation or implementation of an
algorithm in some programming language. The algorithm can be instantiated in any
programming language.
6

7.
Example
• Let us consider a problem of finding maximum element in a finite sequence of integers.
1. Problem:
• Input: A sequence of n integers { a1,a2, ...,an}.
• Output: max{a1,a2, ...,an}.
7
3. Program: in C language
int maxFind(int a[ ], n)
{ max = a[0];
for(int i=0;i<n;i++)
if( max< a[i])
max = a[i];
return max;
}

8.
• An algorithm can be specified in a number of ways such as in English, as
a pseudo-code, or as a computer program.
• There is not any standard method for writing algorithm.
• But, the only requirement is that the specification must provide a precise
description of computational procedure to be followed.
• Pseudo-code is a popular way to express algorithm that uses a mixture of
English phrases and indention to make the steps in the solution explicit.
• Pseudo-code cannot be compiled nor executed, and there are no real
formatting or syntax rules.
• The benefit of pseudo-code is that it enables the programmer to
concentrate on the algorithms without worrying about all the syntactic
details of a particular programming language and easy to convert it into
programs in any language.
Writing Algorithm
8

9.
Step 1: Begin
Step 2: Read numbers a, b and c.
Step 3: If a>b
If a>c
Print a
Else
Print c
Else If b>c
Print b
Else
Print c.
Step 5: End
Example
• Write an algorithm to find the largest among three different numbers
entered by user.
9

10.
Exercise
 Write the algorithms for following tasks:
i. To find sum of first n natural numbers.
ii. To search a key in a list of values with linear search.
iii. To search a key using binary search.
10

11.
Binary Search Algorithm
Iterative Algorithm
binarySearch( A,l,r,k)
{
while(l<r)
{
m = (l+b)/2
if(A[m]== k)
Print " Search Successful !"
return;
else if(k<A[m])
r=m-1
else
l=m+1
}
print "Search Unsuccessful"
}
11
Recursive Algorithm
BinarySearch(A, l, r, key)
{
if(l= = r) //only one element
{
if(key = = A[l])
print " successful Search"
else
print "unsuccessful Search"
}
else
{
m = (l + r) /2 ; //integer division
if(key = = A[m]
print "successful search"
return
else if (key < A[m])
return BinarySearch(A, l, m-1, key) ;
else
return BinarySearch(A, m+1, r, key) ;
}
}

12.
• Input: A number of quantities are provided to an algorithm initially before the
algorithm begins. These quantities are inputs which are processed by the
algorithm.
• Output: An algorithm must have one or more output.
• Definiteness: Each step must be clear and unambiguous that leads to a
specific action.
• Correctness: Correct set of output values must be produced from the each
set of inputs. For the same input data, it must always produce the same
output.
• Finiteness: Algorithm must terminate after finite time or number of steps.
• Effectiveness: Each step must be carried out in finite time.
• Generality: The procedure should be applicable for all problems of the
desired form, not just for a particular set of input values.
Characteristics of Algorithms
12

13.
Example
• The algorithm presented above "To find max element from sequence of integers" has all
properties.
– The algorithm has a sequence of integers as input.
– The output is the largest integer in the sequence.
– Each step is precisely defined, because only assignments, finite loop, and conditional
statements occur.
– The algorithm is correct because it produces max element from the sequence. This can
be examined as the algorithm proceeds initial value of max is updated if new element is
larger than current value.
– The algorithm uses finite number of steps, because it terminates after all the integers in
the sequence has been examined.
– The algorithm can be carried out in a finite amount of time because each step is either a
comparison or an assignment.
– Finally, the algorithm is general, because it can be used to find the maximum element of
any finite sequence of integers.
13

14.
Design of Algorithms
• There are a number of algorithm design techniques that are useful for
effectively dealing with many computational problems.
• Some of these techniques are:
• Divide and Conquer
• Greedy algorithms
• Dynamic Programming
• Backtracking
14

15.
Divide and Conquer Technique
• This algorithm design is recursive technique.
• A problem is broken into a number of sub-problems that are similar to original
problem but smaller in size, solve these sub-problems recursively, and then
combine these solutions to create a solution to the original problem.
• That is, this technique involves three steps:
– Divide the problem into a number of sub-problems.
– Conquer the sub-problems by solving them recursively.
– Combine the solutions to the sub-problems into the solution for the original
problem.
• For example: merge sort algorithm
15

16.
Divide and Conquer Algorithm: Example
Merge Sort Algorithm: It is recursive algorithm having following three steps
to sort an array A[l . . r].
• Divide
– Divide the n-element sequence to be sorted into two sub-sequences of n/2
elements
• Conquer
– Sort the sub-sequences recursively using merge sort. When the size of the
sequences is 1 there is nothing more to do.
• Combine
– Merge the two sorted sub-sequences into single sorted array.
16

17.
Merge Sort
17
MergeSort(A, l, r)
{
If(l<r)
{
m=(l+r)/2
MergeSort(A, l, m)
MergeSort(A, m+1, r)
Merge(A, l, m+1, r)
}
}
while(x<m)
{
B[k] = A[x];
k++;x++;
}
while(y<=r)
{
B[k] = A[y];
k++; y++;
}
For(i =0;i<=r;i++)
{
A[i] = B[i];
}
}
Merge(A, l, m, r)
{
x= l, y=m, k =l;
While(x<m && y<=r)
{
if(A[x] < A[y])
{
B[k] = A[x];
k++; x++;
}
else
{
B[k] = A[y];
k++; y++;
}
}

18.
18Example: division of array a[]= {4, 7, 2, 6, 1, 4, 7, 3, 5, 2, 6}
Merge Sort

19.
19
Merge Sort

20.
Greedy Algorithms
• Greedy technique is simple and efficient algorithm for many optimization
problems.
• This technique works in stages considering one input at a time.
• At each stage, it always makes the choice that looks best at the moment
to move towards the feasible solution of the problem.
• The selection procedure itself is based on some optimization measure.
• For example: design of data compression (Huffman) code, minimum
spanning tree, Dijkstra's algorithm for shortest paths from a single source
20

21.
Example: Finding MST
• A spanning tree for a connected graph G is a tree containing all the
vertices of G.
• A minimum spanning tree ( MST ) in a connected weighted graph is a
spanning tree that has the smallest possible sum of weights of its
edges.
• It represents the cheapest way of connecting all the nodes in G.
• It is not necessarily unique.
• It is applicable any time we want to visit all vertices in a graph at
minimum cost (e.g., wire routing on printed circuit boards, sewer pipe
layout, road planning…).
21

22.
Kruskal's Algorithm finding MST
• We have V as a set of n vertices and E as
set of edges of graph G.
• The idea behind this algorithm is:
• The nodes of the graph are considered as
n distinct partial trees with one node each.
• At each step of the algorithm, two partial
trees are connected into single partial tree
by an edge of the graph.
• While connecting two nodes of partial
trees, minimum weighted arc is selected.
• After n-1 steps MST is obtained.
• Kruskal’s algorithm is a greedy algorithm,
because at each step it adds to the forest
an edge of least possible weight.
• Kruskal's Algorithm
Input - G: weighted connected graph with n
vertices
Output: MST
T = empty tree
for i=1 to n-1
e = any edge in G with smallest weight
but does not form a cycle when
added to T
T = T U {e}
end
22

23.
Example
23
• Find MST from the given graph with Kruskal's
Algorithm.

24.
24
Example Solution
1
Edge
Selection Order

25.
25
Example Solution
1
2
Edge
Selection Order

26.
26
Example Solution
1
2
3
Edge
Selection Order

27.
27
Example Solution
1
2
3
4
Edge
Selection Order

28.
28
Example Solution
1
2
3
4
5
Edge
Selection Order

29.
29
Example Solution
1
2
3
4
5
6
Edge
Selection Order2
7
3
4
5
6
1
7
7
13
8
10
9
• Final MST

30.
Prim's Algorithm: Finding MST
30
2
7
3
4
5
6
1
7
2
7
3
4
5
6
1
7
7
Step: 2Step: 1

31.
31
Prim's Algorithm
2
7
3
4
5
6
1
7
7
13
8
2
7
3
4
5
6
1
7
7
13
8
10
2
7
3
4
5
6
1
7
7
13
8
10
9
2
7
3
4
5
6
1
7
7
13
Step: 3
Step: 5
Step: 4
Step: 6

32.
Dynamic Programming
• Dynamic programming technique is the most powerful technique for solving optimization
problems.
• DP is closely related with divide-and-conquer algorithm, where the problem is partitioned
into independent sub-problems and each sub-problem is solved recursively and then
combine their solutions to solve original problem.
• The DP differs from divide-and-conquer in the way that sub-problems are not independent.
i.e. they share sub-problems and instead of solving sub-problems recursively, DP solves
each sub-problem only once and then saves its answer in the table, thereby avoiding the
work of recomputing the answer each time the sub-problem is encountered. The solution to
main problem is obtained by the solutions of these sub-problems.
• DP ≈ Sub-problems + Re-use
• Dynamic programming is typically applied to optimization problems.
• In such problems there can be many possible solutions. But we wish to find optimal
solution.
32

33.
• Fibonacci Numbers: 1, 1, 2, 3, 5, 8, 13, 21, 34, ...
• Recursive Formula:
o F1=1 and F2=1
o Fn = Fn-1 + Fn-2
• The problem is to compute nth Fibonacci number
• Naive Recursive Algorithm for nth Fibonacci Number
33
Dynamic Programing
fib( n)
{
if ( n == 1 OR n == 2)
return 1;
else
return fib(n-1) + fib(n-2);
}

34.
Recursive Algorithm for nth Fibonacci Number
• Let’s try to calculate the 5th Fibonacci number.
• The representation shown below shows the repetition in the calculation.
• In the following tree we see that calculations of fib(1) is done two times, fib(2) is done 3
times, fib(3) is done 2 times, and so on. So if we somehow eliminate those repeated
compution, we will save the running time.
34
Dynamic Programming : Example
fib(5)
fib(1)fib(2)
fib(2) fib(1)fib(2)fib(3)
fib(4) fib(3)
• fib(5) = (fib(4)+fib(3))
= ((fib(3) + fib(2)) + fib(3))
= (((fib(2) +fib(1)) + fib(2)) +fib(3))
= (((fib(2) +fib(1)) + fib(2)) +(fib(2) + fib(1))
• The problem with algorithm is that we have to
calculate two previous numbers regardless of the
computation that has already been done.
• It results time complexity : T(n) = O(2n ),
which is very slow.

35.
• Dynamic Programming Fibonacci Algorithm:
• It has pretty simple idea: each time we calculate the fibonacci number for some n, memoize it.
i.e. store the value in the table. When next time same computation needs to be done, just use
the table value.
• Analyzing the above algorithm we find that there are no repetition of calculation of the sub-
problems already solved and the running time decreased from O(2n) to O(n).
• This reduction was possible due to the remembrance of the sub-problem that is already solved
to solve the problem of higher size. 35
Dynamic Programming
DynaFibo(n)
{
A[1] = 1, A[2]= 1;
for(i = 3 ; i <=n ; i++)
A[i] = A[i-2] +A[i-1] ;
return A[n] ;
}
Dynamic Fibonacci Idea
• fib(n):
if n is in memo, return memo[n]
if n <= 2 then f =1
else f = fib(n-1) + fib(n-2)
memo[n] = f
return f

36.
Backtracking
• There are problems that can be solved only
by performing an exhaustive search of all
possible solutions.
• Backtracking is a technique used to solve
problems with a large search space, by
systematically trying and eliminating
possibilities.
• Such problems are graph coloring, n-queen
problem, subset sum problems etc.
• A standard example of backtracking would be
going through a maze.
– At some point in a maze, we might have
two options of which direction to go:
36
Portion A
PortionB

37.
Backtracking
Portion B
PortionA
• One strategy would be to try going
through Portion A of the maze.
– If we get stuck before we find our
way out, then we "backtrack" to
the junction.
– At this point in time we know
that Portion A will NOT lead us
out of the maze,
– so we then start searching in
Portion B

38.
Backtracking
 Clearly, at a single junction we could
have even more than 2 choices.
 The backtracking strategy says to try
each choice, one after the other,
 if wevever get stuck, "backtrack"
to the junction and try the next
choice.
 If we try all choices and never found
a way out, then there IS no solution
to the maze.
B
C
A

39.
Backtracking Example
• N-Queen Problem
– There is N X N board and there are N queens.
– We have to find any placement of these queens on the board
so that they do not attack each other.
– A queen attacks on the same column, same row and
diagonally.
• Consider a board of size 4 x 4 and 4 queens.
• Let us consider a queen placed at (1,2).
• The positions (1,0), (1,1), (1,3) are attacking positions with (1,2)
being same row, (0,2), (2,2),(3,2) are attacking positions being
same column and (0,1), (2,3), (0,3),(2,1), (3,0) are diagonally
attacking.
• Find the proper positions of 4 queens in this board.
39
0 1 2 3
0
1 Q
2
3

40.
N-Queen Problem
40
0 1 2 3
0 Q
1 Q
2
3
0 1 2 3
0 Q
0 1 2 3
1 x x Q
0 1 2 3
2 x x x x
0 1 2 3
3 x x x x
0 1 2 3
1 x x Q
0 1 2 3
0 Q
1 Q
2 Q
3
0 1 2 3
2 x Q
0 1 2 3
2 x x x 0 1 2 3
0 Q
1
2
3
0 1 2 3
0 Q

41.
N-Queen Problem
41
0 1 2 3
0 Q
1
2
3
0 1 2 3
0 Q
0 1 2 3
1 x x x Q
0 1 2 3
2 Q
0 1 2 3
3 x x Q
0 1 2 3
0 Q
1 Q
2 Q
3
0 1 2 3
0 Q
1 Q
2 Q
3 Q
Final Positions
(0,1)
(1,3)
(2,0)
(3,2)

42.
• The “analysis algorithm” deals with efficiency of an algorithm. Analysing an
algorithm has come to mean predicting the resources that the algorithm
requires.
• Generally, by analysing several candidate algorithms for a problem, a most
efficient one can be easily identified.
• How can the efficiency of an algorithm be analyzed?
• One measure of efficiency is the time used by a computer to solve using
the algorithm for a specified input size.
• Second measure is the amount of memory required to implement the
algorithm for given input size.
• These efficiency measures studied and analysed in Algorithmic Complexity.
• Algorithmic complexity is a function of input size that refers to the rate at which
required storage or computational time grows as input size increases.
Analysis of Algorithm
42

43.
• There are two main complexity measures of the efficiency of an algorithm: Time
Complexity and Space complexity.
• Time complexity is a function describing the amount of time an algorithm takes with
respect to input to the algorithm.
• Time complexity is expressed in terms of number operations used by the algorithm
for particular input size rather than actual computer time because of difference in
time needed for different computers to perform basic operations.
• The basic operations are: number of arithmetic operation performed, number of
comparisons made, number of times through a loop or number of array elements
are accessed etc.
• The time for the operations independent of input size is ignored, when expressing
time complexity for an algorithm.
43
Analysis of Algorithm

44.
• Usually the resource being considered is running time.
• Although the running time of an implementation of the algorithm would depend upon the
speed of the computer, programming language, compiler etc, these technical issues are
ignored because their effect is constant factor.
• Running time of algorithm varies with size of input or data distribution of input. So, time
complexity analysis refers the rate at which computational time increases as input size
increases.
• Depending upon the distribution of input data, there are three cases:
• Worst case: the maximum amount of time needed by the algorithm for specified size
of input.
• Average case: Expected time or average time of algorithm for specific size of input.
• Best case: minimum amount of time for particular input size. It is the case that
happens for some specific distribution of the data.
44
Analysis of Algorithm

45.
• Worst Case Complexity: This is the worst-case performance of an algorithm, which
means the largest number of operations needed to solve the given problem using the
algorithm on input of specified size.
• Average-case Complexity: This is the average number of operations used to solve the
problem over all inputs of a given input size.
• Best-case complexity is the minimum number operations used to solve the problem
using the algorithm on the input of specified size.
• For example: in case of Linear Search,
for searching some value x, the number of comparison
vary depending upon distribution of numbers in the array.
• Best case occurs when searched item x is
the first term of the list. In this case three
comparisons are needed, one to determine
whether end of array has been reached, one
to compare x and the first term, and one
outside the loop. 45
Analysis of Algorithm
int search(int arr [ ], int n, int x)
{
int i;
for (i=0; i<n; i++) {
if (arr[i] == x) {
location = i;
break; }
}
if(i==n)
printf("not found");
}

46.
• Worst case occurs when searched value x is not in the array.
– Since each iteration of the loop involves two comparisons, one to see whether the
end of list has been reached and other to compare the element x with the term of the
array.
– Finally, one more comparison is made to to exit the loop and one outside the loop.
Hence, total the number of comparisons will be ( 2n + 2).
• Average case: There are n types of possible inputs when x is known to be in the list.
– When element x is the first term in the array, 3 comparisons are needed. If x is the
second term of the list, 5 comparisons are needed.
– In general, if x is the ith term of the list, 2i+1 comparisons are need.
– Hence, average number of comparisons used for this algorithm equals:
3 +5+7+⋯.+ 2𝑛+1
𝑛
+ 1 =
2 1+2+3+⋯.+𝑛 +𝑛
𝑛
+ 1 =
2( 𝑛 𝑛+1 /2)+𝑛
𝑛
+ 1 = 𝑛 + 2
46
Analysis of Algorithm

47.
• Analysis of space complexity of an algorithm is the amount of memory it needs
to run to completion.
• It is concerned with the rate at which required storage space grows as a
function of input size.
• Algorithm has following parts:
• Fixed part: The space required to store certain data/variables, that is
independent of the size of the problem such as instruction space ( i.e.
space for code), space for simple variables, constants and so on.
• Variable part: The space needed by the algorithm whose size depends on
the particular problem instance being solved such as array, recursion stack
etc.
47
Analysis of Algorithm

48.
48
• Example: consider the following algorithm:
• The space needed by n is one word, since it is of type integer.
• The space needed by a is the space needed by variables of type array of
floating point numbers. This is at least n words, since a must be large enough
to hold the n elements to be summed.
• So, Space (sum(a,n))>= (n+3), n for a[ ], 1 for each n, i and s).
Algorithm Sum(a,n)
{
s:=0.0;
for i:=1 to n do
s:=s+a[i];
return s;
}
Analysis of Algorithm

49.
• When analysing the running time or space usage of programs, we usually try to
estimate the time or space as functions of the input size.
• Time complexity is defined as a function from input size to the number of key
operations of computational process.
• Asymptotic analysis estimates the growth of function without worrying about the
hardware and software used to implement an algorithm when input size
increases towards the infinity.
• i.e. asymptotic analysis:
– looks the growth of running time as input size n gets to infinity.
– ignores machine dependent constants
• Asymptotic analysis measures properties of algorithm complexity performance
and/or memory requirements by eliminating constant factors.
• A good rule of thumb is: the slower the asymptotic growth rate, the better the
algorithm. 49
Asymptotic Analysis

50.
Asymptotic Analysis: Example
• Consider algorithms with cost function as: n2 and 2n and let's see how they
grow when input size n gets larger and larger.
• Here, we can see that 2n grows much faster than the n2 as the input size
grows.
• It means that 2n takes much longer time to process the same size of input as
compared to the n2 algorithm.
50
n 1 10 100 1000 104
n2 1 100 104 106 108
2n 2 1024 1.27 x 1030 1.07 x 10301 1.99 x 103010

51.
Big-Oh Notation
• The asymptotic analysis uses different types of notations such as Big-Oh, Big-Omega,
Big-theta etc.
• These notations are mathematical tools that describe the runtime in terms of—how quickly
it grows relative to the input, as the input gets arbitrarily large.
• Big Oh notation is the formal method of expressing the upper bound of an algorithm's
running time. i.e. the longest amount of time it could possibly take for the algorithm to
complete.
• Advantages of Big Oh:
– Estimates the upper bound of the amount of work performed by algorithm. That is, it indicates
worst case time saying that in any case the running time will never exceed this time.
– Helps us to compare different algorithms before deciding which one to implement.
– Provides the growth of a function, focusing on input size without worrying about other constant
factors, thus simplifies the analysis
• Big Oh has following limitations:
− It ignores potential of constant factors in the algorithm.
− It does not try to improve algorithm, only gives the complexity.
51

52.
Big Oh (O) notation
Formal definition
• Let f and g be any two functions defined over set of positive integers. A function
f(n) is big oh of g(n) ), denoted as f(n)=O(g(n)), if there exists two positive
constants c and n0 such that for all n >= n0, f(n) <= c*g(n).
52
• Exmple: 1
Let f(n) = 3n + 2 and g(n) = n. Can we say f(n) = O(g(n)) ?
Solution:
For this we have to find two constants, c >0 and n0>=1
such that f(n) <= c*g(n). i.e. 3n+2 <= c.n, Ɐn>=n0
We can choose any value for c > 0 and n0>=1.
Here, we see that if we put c=4, the inequality holds good
Ɐn >=2, i.e. 3n+2 <= 4n, Ɐn >=2.
Thus, we can say f(n) = O(g(n)). So, g(n) upper bound of f(n).
NOTE: Any function greater than g(n) will also be upper bound of
f(n).

53.
53
Big Oh (O) notation
• Example 2
f(n)=5n3+3n2+4 and g(n) =12n3 Then, is f(n) = O(n3)?
Solution,
Let us find two constants c>0 and n0 >= 1 such that f(n) <= c * g(n), for all n>n0.
Let us take c = 12 and n0 = 1.
n f(n) g(n) f(n) <= c*g(n)
1 12 12 True
2 56 96 True
3 166 324 True
4 372 768 True
Therefore, for all n >= n0 =1, c = 12, f(n) <= c*g(n) => f(n) = O(g(n)).

54.
• Example: 3
If f(x)= 5x2 + 4x + 2 find big-O of f(x).
Solution,
The simple idea is that just take highest degree term ignoring leading
constant and low order terms.
i.e. O(f(x)) = O(x2).
This happens because when x → ∞, the effect of the constant and low
order term becomes insignificant.
54
Big Oh (O) notation
Alternatively we can see,
Here,
f(x) = 5x2 + 4x + 3 <= 5x2 + 4x2 + 2x2
= 11x2 , for all x>=1
f(x) <= c * g(x), where c =11,
g(x) = x2 and x0 = 1.
Therefore, f(x) = O(x2 ).
x f(x) 5x2 4x 2
10 542 500 (92.25%) 40 (7.38%) 2 (0.0037%)
100 50402 50000 (99.20%) 400 (0.0079%) 2(0.00003%)
1000 5004002 5000000 (99.92%) 4000 (0.00079%) 2 (0.000%)
10000 500040002 500000000 (99.999%) 40000 (0.0000%) 2 (0%)

55.
Big Omega ( Ω ) notation
• Big Omega notation gives asymptotic lower bound of algorithm. That is, it gives best case
time for the algorithm. This means, in any case we can never assume better than this time.
• Formal definition
Let f and g be any two functions defined over set of positive integers. A function f(n) is big
Omega of g(n) ), denoted as f(n)= Ω (g(n)), if there exists two positive constants c and n0
such that for all n >= n0, f(n) >= c*g(n).
• The above relation says that g(x) is a lower bound of f(x).
55
Example:
If f(n)= 3n2 + 4n + 7, find big omega of f(n).
Here,
f(n)= 3n2 +4n + 7 >= 3n2 for all n>= 1.
i.e. f(n) >= c * g(n), where c = 3, n0=1
and g(n) =n2 .
Therefore, f(n) = Ω (g(n).
Note: any function smaller than g(n) is also lower
bound of f(n).

56.
Big Theta ( Θ ) notation
• When we need asymptotically tight bound then we use Θ notation. That is, it gives average
case.
• Formal definition
Let f and g be any two functions defined over set of positive integers. A function f(n) is big
Theta of g(n) ), denoted as f(n)= Θ (g(n)), if there exists three positive constants c1, c2 and
n0 such that for all n >= n0, c1*g(n) <= f(n) <= c2*g(n).
• The above relation says that f(x) is order of g(x)
56
• Example: f(n) = 3n2 + 4n + 7 and g(n) = n2 , then
prove that f(n) = (g(n)).
Proof:
let us choose c1, c2 and n0 values as 14, 1 and 1 respectively
then we can have,
f(n) <= c1*g(n), n>=n0 as 3n2 + 4n + 7 <= 14*n2 , and
f(n) >= c2*g(n), n>=n0 as 3n2 + 4n + 7 >= 1*n2
for all n >= 1(in both cases).
So c2*g(n) <= f(n) <= c1*g(n) is trivial.
Hence f(n) = Θ(g(n)).

57.
Analysis of Algorithm
Example 1 find the time and space complexity
of following algorithm.
A( n)
{
int i;
for(i=1 to n)
print " Ravi"
}
• Here, print line will be printed n times. So,
T(n) = O(n).
• Space complexity = O(1)
Example: 2
A( n)
{
int i, j;
for(i =1 to n)
for(j =1 to n)
print " Ravi"
}
• Here, for each iteration of i inner loop runs
n times. So, Time complexity T(n) = O(n2)
• Space complexity = O(1)
57

58.
Analysis of Algorithms
Example 3:
fib(n)
{
a = 0, b= 1, f=1 ;
for(i = 2 ; i <=n ; i++)
{
f = a+b ;
a=b ;
b=f ;
}
return f ;
}
• Here, all the operations take constant time and
loop iterates upto n times . So, T(n) = O(n).
• Space complexity = O(1)
58
Example 4: Algorithm
BubbleSort(A, n)
{
for(i = 0; i <n-1; i++)
{
for(j = 0; j < n-i-1; j++)
{
if(A[j] > A[j+1])
{ temp = A[j];
A[j] = A[j+1];
A[j+1] = temp; }
}
}
}
• Time Complexity: Inner loop executes for (n-1) times
when i=0, (n-2) times when i=1 and so on: Time
complexity = (n-1) + (n-2) + (n-3) + …………………………. +2
+1 = O(n2)
• Space Complexity = O(n)

59.
Homework #3
1. What is an algorithm? Write the properties of an algorithm.
2. What are two important resources used for analysis of an algorithm?
Explain.
3. What is asymptotic analysis of an algorithm?
4. Explain divide and conquer paradigm of algorithm design with example.
5. What do you mean greedy strategy? Explain with example.
6. Explain dynamic programming concept.
7. Define Big-O with its significance in algorithmic analysis.
8. Define Big – Omega and Big-Theta.
9. Explain N-Queen problem in the context of backtracking algorithm
design.
59

60.
10. Analyze the following algorithms to find time complexity and space
complexity.
60
Homework #3
Ex : 1
int prod( int n)
{
int sum =0;
for(j=0;j<n;j++)
for(k=0;k<n;k++)
sum += j *k;
return sum;
}
Ex : 2
int calc( int n)
{
int sum =0;
for(j=0;j<n;j++)
sum += j;
for(k=0;k<n;k++)
sum -= k;
return sum;
}

61.
Thank You !
61