1.
1
SYLLABUS
• Introduction to algorithms 4HRS
• Divide and conquer 7HRS
• Greedy Method 7HRS
• Dynamic Programming 6HRS
• Backtracking 6HRS
• Branch and Bound 6HRS
• Internet Algorithm 5HRS
PREREQUISITE:
DATA STRUCTURES, DISCRETE MATHEMATICS
C++ or JAVA
Shiwani Gupta

2.
2
Text Books
• 1. Ellis horowitz, Sarataj Sahni, S. Rajsekaran.”Fundamentals of computer
Algorithms” University press.
• 2. Anany V. Levitin ”Introduction to the Design and Analysis of Algorithms”
Pearson Education publication, Second Edition.
• 3. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, "Introduction to
Algorithms", 2nd Edition, MIT Press/McGraw Hill, 2001.
• 4. Michael Goodrich & Roberto Tamassia, “Algorithm design foundation,
analysis and internet examples”, Second Edition, Wiley s.
• 5. S. Baase, S and A. Van Gelder, "Computer Algorithms: Introduction to
Design and Analysis",3rd edition. Addison Wesley, 2000.
• 6. Kenneth Berman, Jerome Paul “Algorithm: sequential, parallel and
distributed” Cengage Learning.
• 7. Mark Allen Weiss, “Data Structure & Algorithm Analysis in C++”, Third
Edition, Pearson Education.
Shiwani Gupta

3.
3
Kindly Revise
DSF
• Sorts (Insertion,
Selection, Bubble, Quick,
Merge, Heap)
• Search (Linear, Binary,
Hashing)
• Tree (Huffman, BST, B-
tree, B+ tree, AVL tree)
DSGT
• Hamiltonian Graph
• TSP
• Spanning Tree
• Graph Coloring
• Warshall’s Algorithm
• Fibonacci-Recurrence
Shiwani Gupta

4.
MODULE 1
INTRODUCTION TO ANALYSIS OF
ALGORITHMS

5.
5
What is an algorithm?
An algorithm is
a sequence of unambiguous instructions
for solving a computational problem,
i.e., for obtaining a required output
for any legitimate input
in a finite amount of time.
Shiwani Gupta

6.
6
Features of Algorithm
• Input : zero or more valid inputs are clearly specified
• Output: produce at least 1 correct output given valid input
• Definiteness: clearly and unambiguously specified
instructions
• Finiteness : terminates after finite steps for all cases
• Effectiveness: steps are sufficiently simple and basic
Shiwani Gupta

7.
7
A Brief History of Algorithms
• According to the Oxford English Dictionary, the
word algorithm is a combination of the Middle
English word algorism with arithmetic.
The word algorism derives from the name of the
Arabic mathematician Al-Khwarizmi.
Al-Khwarizmi wrote a book on solving equations
from whose title the word algebra derives.
• It is commonly believed that the first algorithm was
Euclid’s Algorithm for finding the greatest common
divisor of two integers, m and n (m ≥n).
Shiwani Gupta

8.
8
Rules for writing an Algorithm
• Head
Algorithm name (p1,p2,…..,pn)
//Problem Description:
// Input:
// Output:
• Body {includes programming constructs or assignment statements}
Compound statements enclosed in { }
Single line comments //
Identifier can be combination of alphanumeric string beginning by
letter
Use assignment operator
Boolean, Logical, Relational operators
Array indices [ ] begin with 0
Input and Output using read(val) and write(“Hello”)
if (condition) then statement
if (condition) then statement else statement
Shiwani Gupta

9.
9
while (cond) do
{
stmt 1
stmt 2
:
:
stmt n
}
for var val1 to valn do
{
stmt 1
stmt 2
:
:
stmt n
}
Rules for writing an Algorithm
repeat
{
stmt 1
stmt 2
:
:
stmt n
} until (condition)
break stmt to exit from inner loop
return stmt to return control from one
point to another
Shiwani Gupta

10.
10
ALGORITHM DESIGN AND ANALYSIS PROCEDURE
Shiwani Gupta

11.
11
• Brute Force
• Divide and Conquer / Decrease and Conquer
– Merge Sort / Binary Search
• Greedy Method
– Kruskal algorithm for minimal spanning tree
• Dynamic Programming
– All pair shortest path
• Search and Enumeration (Search algo, B&B, Backtracking)
– Graph Problems
• Probabilistic and Heuristics and Genetic
Classification by Design Paradigm
Shiwani Gupta

12.
12
Classification of Algorithm by Implementation
• Recursion / Iteration
– Tower of Hanoi, Fibonacci
• Logical
– Algorithm = logic + control
• Serial / Parallel or Distributed
– Sorting, Iterative
• Deterministic / Non Deterministic
– Exact decision / Guess via heuristics
• Exact / Approximate
– Approximate algorithms are useful for hard problems
Shiwani Gupta

13.
13
Expressing Algorithms
• Natural Language
– Verbose and ambiguous
• Pseudocode
– Structured, avoids ambiguity of natural languages
• Flowchart
– Structured, avoids ambiguity of natural languages
• Programming Languages
– Express in a form executable by computer
Shiwani Gupta

14.
14
Analysis of Algorithm
• Determine run-time of a program as function of
input –TIME COMPLEXITY
• Determine total or maximum memory required for
program data – SPACE COMPLEXITY
• Determine total size of program code
• Determine whether program correctly computes
desired result
• Determine complexity of program
– Ease of reading, understanding and modification
• Determine robustness of program
– Dealing with unexpected and erroneous inputShiwani Gupta

15.
15
Space Complexity
• The space S(p) needed by an algorithm is the sum
of a fixed part and a variable part
• The fixed part c includes space for
– Instructions
– Variables
– Identifiers
– Constants
• The variable part Sp includes space for
– Variables whose size is dependant on the particular
problem instance being solved
– Recursion stack space S(p) = c + SpShiwani Gupta

16.
16
Algorithm abc (a, b, c)
{
return a+b+b*c+((a+b-c)/(a+b)+4.0;
}
For every instance 3 computer words require to store variables: a, b, c
Sp () = 3
Algorithm Sum (a, n)
{
s:=0.0;
for i = 1 to n do
s := s + a[i];
return s;
}
Every instance needs to store array a[] and n
Space needed to store n = 1 word
Space needed to store a[] = n floating point words
Space needed to store i and s = 2 words
Sp (n) = n+3
Shiwani Gupta

17.
17
Time Complexity
• The time complexity of a problem is
– The number of steps that it takes to solve an instance of
the problem as a function of the size of the input
(usually measured in bits), using the most efficient
algorithm.
• The time needed by an algorithm T(p) is the sum of
a fixed part and a variable part
– The fixed part includes compile time c which is
independent of problem instance.
– The variable part is the run time tp which is dependent
on problem instance.
T(p) = c + tpShiwani Gupta

18.
18
Analyzing Running Time
1. n = read input from user
2. sum = 0
3. i = 0
4. while i < n
5. number = read input from user
6. sum = sum + number
7. i = i + 1
8. mean = sum / n
T(n), or the running time of a particular algorithm on input of size n, is
taken to be the number of times the instructions in the algorithm are
executed.
Example pseudo code algorithm illustrates the calculation of the mean
(average) of a set of n numbers:
Number of times executed
1
1
1
n+1
n
n
n
1
The computing time for this algorithm in
terms on input size n is: T(n) = 4n + 5.Shiwani Gupta

19.
19
Sr.
No.
Statements S/E Freq. Total
1 Algorithm Sum(a, n ) 0 - 0
2 { 0 - 0
3 S=0.0; 1 1 1
4 for i=1 to n do 1 n+1 n+1
5 s=s+a[i]; 1 n n
6 return s; 1 1 1
7 } 0 - 0
Sr.
No.
Statements S/E Freq. Total
1 Algorithm Add(a, b, c, n, m ) 0 - 0
2 { 0 - 0
3 for i=1 to n do 1 n+1 n+1
4 for j=1 to m do 1 n(m+1) n(m+1)
5 c[i,j]:=a[i,j]+b[i,j]; 1 nm nm
6 } 0 - 0
Shiwani Gupta

20.
20
Best, average, worst-case Efficiency
• Worst case:
– Efficiency (# of times the basic operation will be executed) for the worst case
input of size n, for which
– The algorithm runs the longest among all possible inputs of size n.
• Best case:
– Efficiency (# of times the basic operation will be executed) for the best case
input of size n, for which
– The algorithm runs the fastest among all possible inputs of size n.
• Average case:
– Efficiency (#of times the basic operation will be executed) for a typical/random
input
– NOT the average of worst and best case.
Shiwani Gupta

21.
21
Algorithm’s Performance (Asymptotics)
Used to formalize that an algorithm has running
time or storage requirements that are
``never more than'' ,
``always greater than” , or
``exactly'' some amount
Less than equal to (“≤”)
Greater than equal to (“≥”)
Equal to (“=“)
Shiwani Gupta

22.
22
Big Oh (O) Notation
Asymptotic Upper Bound
Definition 1: Let f(n) and g(n) be two functions. We write:
f(n) = O(g(n)) or f = O(g)
(read "f of n is big oh of g of n" or "f is big oh of g")
if there is a positive integer C such that f(n) <= C * g(n) for all positive integers n>n0.
The basic idea of big-Oh notation is this: Suppose f and g are both real-valued
functions of a real variable x. If, for large values of x, the graph of f lies closer to the
horizontal axis than the graph of some multiple of g, then f is of order g, i.e., f(x) =
O(g(x)). So, g(x) represents an upper bound on f(x).
Shiwani Gupta

23.
23
Common plots of O( )
O(2n)
O(n3 )
O(n2)
O(nlogn)
O(n)
O(√n)
O(logn)
O(1)
Shiwani Gupta

24.
24
Omega (Ω) Notation
Asymptotic Lower Bound
Definition 1: Let f(n) and g(n) be two functions. We write:
f(n) = Ω(g(n)) or f = Ω(g)
(read "f of n is omega of g of n" or "f is omega of g")
if there is a positive integer C such that f(n) >= C * g(n) >= 0 for all positive integers n>n0.
The basic idea of omega notation is this: Suppose f and g are both real-valued
functions of a real variable x. If, for large values of x, the graph of some multiple of g
lies closer to the horizontal axis than the graph of f, then f is of order g, i.e., f(x) =
Ω(g(x)). So, g(x) represents an lower bound on f(x).
Shiwani Gupta

25.
25
Theta (Θ) Notation
Asymptotic Tight Bound
Definition 1: Let f(n) and g(n) be two functions. We write:
f(n) = Θ (g(n)) or f = Θ (g)
(read "f of n is theta of g of n" or "f is theta of g")
if there is a positive integer C such that
C2 * g(n) >= f(n) >= C1 * g(n) >= 0 for all positive integers n.
The basic idea of theta notation is this: Suppose f and g are both real-valued functions
of a real variable x. If, for large values of x, the graph of some multiple of g lies closer
to the horizontal axis than the graph of f and the graph of f lies closer to the horizontal
axis than some other multiple of g, then f is of order g, i.e., f(x) = Ω(g(x)). So, g(x)
represents a tight bound on f(x). Thus g(x) is both upper and lower bound on f(n).
Thus, (f) = O(f)  (f)
Shiwani Gupta

26.
26
Compare n and (n+1)/2
lim( n / ((n+1)/2 )) = 2,
same rate of growth
(n+1)/2 = Θ(n)
rate of growth of a linear function
Compare n2 and n2+ 6n
lim( n2 / (n2+ 6n ) )= 1
same rate of growth.
n2+6n = Θ(n2)
rate of growth of a quadratic function
Compare log n and log n2
lim( log n / log n2 ) = 1/2
same rate of growth.
log n2 = Θ(log n)
logarithmic rate of growth
Θ(n3): n3
5n3+ 4n
105n3+ 4n2 + 6n
Θ(n2): n2
5n2+ 4n + 6
n2 + 5
Θ(log n): log n
log n2
log (n + n3)
Shiwani Gupta

27.
27
Input
Size:
n
(1) log n n n log n n² n³ 2ⁿ n!
constant log linear n-log-n quadratic cubic exponential factorial
Scanning
Array
Elements
Performin
g Binary
Search
Operation
Performing
Sequential
Search
Operation
Sorting
Elements
using Merge
Sort or Quick
Sort
Scanning
Matrix
Elements
Performing
Matrix
Multiplicatio
n
The Towers
of Hanoi
problem
Traveling
Salesman
Problem by
Brute Force
Search
5 1 3 5 15 25 125 32 120
10 1 4 10 33 100 10³ 10³ 3628800
100 1 7 100 664 104 106 1030 9.33e+157
1000 1 10 1000 104 106 109 10300 4.02e+2567
10000 1 13 10000 105 108 1012 103000 2.84e+35659
Generalizing Running Time (Basic Efficiency classes)
Shiwani Gupta

28.
28
Running Time Calculations - Loops
for (j = 0; j < n; ++j) {
// 3 atomics
}
• Complexity = (3n) =
(n)
- Loops with Break
for (j = 0; j < n; ++j) {
// 3 atomics
if (condition) break;
}
• Upper bound = O(4n) = O(n)
• Lower bound = (4) = (1)
• Complexity = O(n) Shiwani Gupta

29.
29
- Loops in
Sequence
for (j = 0; j < n; ++j) {
// 3 atomics
}
for (j = 0; j < n; ++j) {
// 5 atomics
}
• Complexity = (3n + 5n)
= (n)
- Nested
Loops
for (j = 0; j < n; ++j) {
// 2 atomics
for (k = 0; k < n; ++k) {
// 3 atomics
}
}
• Complexity = ((2 + 3n)n) = (n2)Shiwani Gupta

30.
30
-Dependent
Quadratic
1 i=1
2 loop (i<=n)
1 j=1
2 loop (j<=i)
1 appl code
2 j=j+1
3 i=i+1
Complexity = ((3n(n+1)/2)+3n+3) = (n2)
-
Quadratic
1 i=1
2 loop (i<=n)
1 j=1
2 loop (j<=n)
1 appl code
2 j=j+1
3 i=i+1
Complexity = ((3n2+4n+2) = (n2)
Shiwani Gupta

31.
31
-Multiply
Loop
1 i=1
2 loop (i<n)
1 appl code
2 i = i*2
Complexity = (3log2 n + 2) = (log2 n)
- Divide
Loop
Logarithmic Loops
1 i=n
2 loop (i>=1)
1 appl code
2 i = i/2
Complexity = (3log2 n + 5) = (log2 n)
Shiwani Gupta

32.
32
- Consecutive
Statements
for (i = 0; i < n; ++i) {
// 1 atomic
if(condition) break;
}
for (j = 0; j < n; ++j) {
// 1 atomic
if(condition) break;
for (k = 0; k < n; ++k)
{
// 3 atomics
}
if(condition) break;
}
• Complexity =
O(2n) + O((2+3n)n)
• = O(n) + O(n2) = ??
• = O(n2)
Shiwani Gupta

33.
33
- If-then-else
if(condition)
i = 0;
else
for ( j = 0; j < n; j++)
a[j] = j;
• Complexity
= O(1) +
max ( O(1), O(N))
= O(1) + O(N)
= O(N)
- Sequential Search
• Given an unsorted vector a[], find if the
element X occurs in a[]
for (i = 0; i < n; i++) {
if (a[i] == X) return true;
}
return false;
• Complexity = O(n)
Shiwani Gupta

34.
34
Randomized Algorithms / Monte Carlo
Algorithms / Stochastic Methods
• A randomized algorithm is just one that depends on random numbers
– to help find a solution to a problem
– to improve a solution to a problem
• Avoids worst-case behavior and can (probabilistically) guarantee
average case behavior
• Provides efficient approximate solutions to intractable problems
• Perhaps the best way of generating “random” numbers is by the
linear congruential method:
– r = (a * r + b) % m;
where a and b are large prime numbers, and m is 232 or 264
– The initial value of r is called the seed
– If you start over with the same seed, you get the same sequence
of “random” numbers
Shiwani Gupta

35.
35
Randomized Algorithm
• Execution time may vary from run to run
• For different inputs there may be different outcomes on each run
• For same inputs there may be different outcomes on each run
• Advantages
– Simple to implement
– Many times efficient than traditional algorithms
• Disadvantages
• Small degree of error may be dangerous for some applications
• Not always possible to obtain better results using randomized algorithms
Shiwani Gupta

36.
36
Types of Randomized Algorithms
• Monte Carlo (Different outcome on each execution for same input)
– Running time bounded by input size (fixed), but answer allows
small probability of error
– Decision problems: If there is no solution, always returns “no”. If
there is a solution, finds it with some probability >= ½.
– Value problems: run for a bounded number of steps, produce an
answer that is correct approximation with a bounded probability
(function of number of steps)
• Las Vegas (Same outcome on each execution for same input)
– Guaranteed to produce correct answer, but running time is
probabilistic
• Atlantic City
– Running time bounded by input
– Can return either “yes” or “no” regardless of correct answer.
Correct with probability >= 2/3.
Shiwani Gupta

37.
37
Mathematical Background for non
recursive algorithm analysis
1. Decide on a parameter(s) indicating an input’s size.
2. Identify algorithms basic operation.
3. Check whether the number of times basic operation is executed
depends only on size of input. Investigate worst, average and best
case efficiencies.
4. Find no. of times the basic operation is executed.
5. Either find a closed formula for count or its order of growth.
// Largest element in an array
ALGORITHM
MaxElement(A[0...n-1])
Maxval A[0]
for i 1 to n-1 do
if A[i]>maxval
maxval A[i]
return maxval
Shiwani Gupta

38.
38
Mathematical Background for recursive
algorithm analysis
1. Decide on a parameter(s) indicating an input’s size.
2. Identify algorithms basic operation.
3. Check whether the number of times basic operation is executed can
vary on different inputs of same size. Investigate worst, average and
best case efficiencies.
4. Set up a recurrence relation with an appropriate initial condition for
no. of times the basic operation is executed.
5. Solve recurrence or ascertain order of growth.
// factorial for an arbitrary nonnegative no.
ALGORITHM F(n)
if n=0 return 1
else return F(n-1)*n
Shiwani Gupta

39.
39
Algorithm: Fibonacci
Algorithm 1 fib(n)
if n = 0 then
return (0)
if n = 1 then
return (1)
return (fib(n − 1) + fib(n − 2))
Algorithm 2: fib(n)
comment: Initially we create an array A[0: n]
A[0] ← 0,A[1] ← 1
for i = 2 to n do
A[i] = A[i − 1] + A[i − 2]
return (A[n])
Recurrence Relation of Fibonacci Number fib(n):
{0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, …}
Shiwani Gupta