This document provides an overview of algorithms and asymptotic analysis. It defines key terms like algorithms, complexity analysis, and input/output specifications. It discusses analyzing time and space complexity, and introduces asymptotic notations like Big-O, Big-Omega, and Big-Theta to classify algorithms based on how their running time grows relative to the input size. Common algorithm classes like logarithmic, linear, quadratic, and exponential functions are presented. Examples of linear and binary search algorithms are provided to illustrate algorithm specifications and complexity analyses.

1. Data Structures & Algorithms
Topic : Asymptotic Notations & Complexity
Analysis

2. Contents
• Basic Terminology
• Complexity of Algorithm
• Asymptotic Notations
• Linear Search and Binary Search
• Review Questions

3. Basic Terminology
• Algorithm: is a finite step by step list of well-defined
instructions for solving a particular problem.
• Complexity of Algorithm: is a function which
gives running time and/or space requirement in terms of
the input size.
• Time and Space are two major measures of efficiency
of an algorithm.

4. Algorithm
Algorithm
Specification of Input
(e.g. any sequence of
natural numbers)
Specification of Output
as a function of Input
(e.g. sequence of sorted
natural numbers)

5. Characteristics of Good Algorithm
• Efficient
• Running Time
• Space used
• Efficiency as a function of input size
• Size of Input (5, 55555555555)
• Number of Data elements

6. Time-Space Tradeoff
• By increasing the amount of space for storing the
data, one may be able to reduce the time needed for
processing the data, or vice versa.

7. Complexity of Algorithm
• Time and Space used by the algorithm are two
main measures for efficiency of any algorithm M.
• Time is measured by counting the number of key
operations.
• Space is measured by counting the maximum of
memory needed by the algorithm.

8. • Complexity of Algorithm is a function
f(n) which gives running time and/or space
requirement of algorithm M in terms of the
size n of the input data.
• Worst Case: The maximum value of f(n) for
any possible input.
• Average Case: The expected or average value
of f(n).
• Best Case: Minimum possible value of f(n).

9. Rate of Growth
• The rate of growth of some standard functions
g(n) is:
log2n < n < nlog2n < n2 < n3 < 2n

10. Asymptotic Notations

11. Asymptotic Notations
• Goal: to simplify analysis of running time .
• Useful to identify how the running time of an
algorithm increases with the size of the input in the
limit.

12. Asymptotic Notations
Special Classes of Algorithms
• Logarithmic: O(log n)
• Linear: O(n)
• Quadratic: O(n2)
• Polynomial: O(nk), k >= 1
• Exponential: O(an), a > 1

13. Big-Oh (O) Notation
• Asymptotic upper bound
• f(n) = O (g(n)), if there exists constants c
and n0 such that,
f(n) <= c g(n) for all n >= n0
• f(n) and g(n) are functions over non-negative
integers.
• Used for Worst-case analysis.

14. Big-Oh (O) Notation
• Simple Rule:
Drop lower order terms and constant factors.
Example:
• 50n log n is O(n log n)
• 8n2 log n + 5 n2 + n is O(n2 log n)

15. Big-Omega (Ω) Notation
• Asymptotic lower bound
• f(n) = Ω (g(n)), if there exists constants c
and n0 such that,
c g(n) <= f(n) for all n >= n0
• Used to describe Best-case running time.

16. Big-Theta (Ө)Notation
• Asymptotic tight bound
• f(n) = Ө (g(n)), if there exists constants c1,
c2 and n0 such that,
c1 g(n) <= f(n) <= c2 g(n) for n >= n0
• f(n) = Ө (g(n)), iff f(n) = O(g(n)) and
f(n) = Ω (g(n))

17. Little-Oh (o) Notation
• Non-tight analogue of Big-Oh.
• f(n) = o (g(n)), if for every c there exists n0
such that,
f(n) < c g(n) for all n >= n0
• f(n) = o (g(n)), iff
f(n) = O (g(n)) and f(n) ≠ Ω (g(n))
• Used for comparisons of running times.

18. Questions

19. Review Questions
• When an algorithm is said to be better than the
other?
• Can an algorithm have different running times on
different machines?
• How the algorithm’ running time is dependent on
machines on which it is executed?

20. Review Questions
Find out the complexity:
function ()
{
if (condition)
{
for (i=0; i<n; i++) { // simple statements}
}
else
{
for (j=1; j<n; j++)
for (k=n; k>0; k--) {// simple statement}
}
}

21. Review Questions
Find out the complexity:
• void complex (int n)
{
for (int i=1; i*i<=n; i++)
for (int j=1; j*j<=n; j++)
{
cout<<” * ”;
}
}

22. Review Questions
Find out the complexity:
• void more_complex (int n)
{
for(int i=1; i<=n/3; i++)
{
for(int j=1; j<=n; j+=4)
{
cout<<” * ”;
break;
}
break;
}
}

23. Searching

24. Searching
1. Linear Search:
• Compares the item of interest with each element
of Array one by one.
• Traverses the Array sequentially to locate the
desired item.

25. Linear Search Algorithm
• LINEAR (DATA, N, ITEM, LOC)
1. [Insert ITEM at the end of DATA] Set Data [N]= ITEM.
2. [Initialize Counter] Set LOC=0.
3. [Search for ITEM.]
Repeat while DATA [LOC] != ITEM
Set LOC = LOC +1.
[End of loop.]
4. [Successful?] if LOC = N, then Print “ITEM not found”.
else return LOC.
5. Exit.

26. 2. Binary Search
• BINARY ( DATA, LB, UB, ITEM, LOC )
1. [Initialize Segment Variables]
Set BEG = LB, END = UB and MID = INT ((BEG+END)/2).
2. Repeat Steps 3 and 4 while BEG < END and DATA [MID] != ITEM.
3. If ITEM < DATA[MID], then:
Set END = MID - 1.
Else:
Set BEG = MID + 1.
[End of if Structure.]
4. Set MID = INT ((BEG+END)/2).
[End of Step 2 Loop.]
5. If DATA [MID] = ITEM, then: Print “Item Found at ” LOC
Else:
Set LOC = NULL.
[End of if structure.]
6. Exit.

27. Limitations of Binary Search
• Although the complexity of Binary Search is
O (log n), it has some limitations:
1. the list must be sorted
2. one must have direct access to the middle element
in any sub-list.