Best for notes

1. Design & Analysis of Algorithms
(KCS 503)
UNIT-1
Asymptotic Notations
From:
Asif Khan
Department of CSE
GNIOT, Greater Noida
Reference:
Thomas H. Corman et al.
The MIT Press

2. Asymptotes
Asymptotes are the tangent of a curve at infinity.
i.e. The line that touch the curve at infinity.

3. Asymptotic Notations
The order of growth of the running time of an
algorithm, defined earlier, gives a simple
characterization of the algorithm’s efficiency and also
allows us to compare the relative performance of
alternative algorithms.
When we look at input sizes large enough to make only
the order of growth of the running time relevant, we will
focus at the asymptotic efficiency of algorithms.
That is, we are concerned with how the running time of
an algorithm increases with the size of the input, as the
size of the input increases without bound.
Usually, an algorithm that is asymptotically more
efficient will be the best choice for all but very small
inputs.

4. Asymptotic Notations
Asymptotic notation: The notations we use to describe
the asymptotic running time of an algorithm are defined
in terms of functions whose domains are the set of
natural numbers N = { 0, 1, 2, ……}. Such notations are
convenient for describing the worst-case running-time
function T(n), which usually is defined only on integer
input sizes.
When we use asymptotic notation to apply to the
running time of an algorithm, we need to understand
which running time we mean. Sometimes we are
interested in the worst-case running time.
Often, however, we wish to characterize the running
time no matter what the input.

5. Asymptotic Notations
O-Notation: The O-notation (Big Oh Notation)
asymptotically bounds a function from above. When we
have only an asymptotic upper bound, we use O-
notation.
For a given function g(n), we denote by O(g(n)), the set
of functions.
O(g(n)) = { f(n) : there exist positive constants c and n0
such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }
We use O-notation to give an upper bound on a
function, to within a constant factor.

6. Asymptotic Notations
Figure 1 shows the intuition behind O-notation. For all
values n at and to the right of n0, the value of the function
f(n) is on or below cg(n).
Figure 1 ( O-Notation )

7. Asymptotic Notations
Example: Let f(n) = 2n2 + 3n + 5
f(n) = 2n2 + 3n + 5 ≤ 3n2
As f(n) ≤ cg(n), where c = 3 and g(n) = n2
for all n ≥ 5
So f(n) = 2n2 + 3n + 5 = O( n2 )

8. Asymptotic Notations
Ω-Notation: The Ω notation (Big Omega Notation)
asymptotically bounds a function from below. When we
have only an asymptotic lower bound, we use Ω-
notation.
For a given function g(n), we denote by Ω(g(n)), the set
of functions.
Ω(g(n)) = { f(n) : there exist positive constants c and n0
such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }
We use Ω-notation to give a lower bound on a function,
to within a constant factor.

9. Asymptotic Notations
Figure 2 shows the intuition behind Ω-notation. For all
values n at and to the right of n0, the value of the function
f(n) is on or above cg(n).
Figure 2 (Ω-Notation )

10. Asymptotic Notations
Example: Let f(n) = 2n2 + 3n + 5
f(n) = 2n2 + 3n + 5 ≥ 2n2
As f(n) ≥ cg(n), where c = 2 and g(n) = n2
for all n ≥ 1
So f(n) = 2n2 + 3n + 5 = Ω( n2 )

11. Asymptotic Notations
θ-Notation: The θ notation asymptotically bounds a
function from above and below, so it is called tightly
bound.
For a given function g(n), we denote by θ(g(n)), the set
of functions.
θ(g(n)) = { f(n) : there exist positive constants c1, c2 and
n0 such that 0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n) for all n ≥ n0 }
A function f(n) belongs to the set g(n), if there exist
positive constants c1 and c2 such that it can be
“sandwiched” between c1g(n) and c2g(n), for
sufficiently large n.

12. Asymptotic Notations
Figure 3 shows the intuition behind θ-notation. For all
values n at and to the right of n0, the value of the function
f(n) is in between c1g(n) and c2g(n),
Figure 3 (θ -Notation )

13. Asymptotic Notations
Example: Let f(n) = 2n2 + 3n + 5
2n2 ≤ 2n2 + 3n + 5 ≤ 3n2
As c1g(n) ≤ f(n) ≤ c2g(n), for all n ≥ 5
where c1 = 2, c2 = 3 and g(n) = n2
So f(n) = 2n2 + 3n + 5 = θ( n2 )

14. Asymptotic Notations
o-Notation: The asymptotic upper bound provided by
O-notation may or may not be asymptotically tight. The
bound 2n2 = O(n2) is asymptotically tight, but the bound
2n = O(n2) is not. We use o-notation to denote an upper
bound that is not asymptotically tight. We formally
define o(g(n)) (“little-oh of g of n”) as the set
o(g(n)) = { f(n) : for every positive constants c, there
exist a positive constant n0 such that 0 ≤ f(n) < c g(n) for
all n ≥ n0 }
For example, 2n = o(n2) but 2n2 ≠ o(n2)

15. Asymptotic Notations
Example: Let f(n) = 2n2 + 3n + 5
f(n) = 2n2 + 3n + 5 < c n3 --------(1)
For every positive constant c there exist a positive
constant n0 such that eqn (1) is true.
For larger values of c, eqn (1) is obviously true.
For smaller values of c for ex 1/1000
2n2 + 3n + 5 < n3 /1000 for every n ≥ 2000
As f(n) < cg(n), where c = 1000 and g(n) = n3
for all n ≥ 2000
So f(n) = 2n2 + 3n + 5 = o( n3 )

16. Asymptotic Notations
ω-Notation: By analogy, ω-notation is to Ω-notation as
o-notation is to O-notation. We use ω-notation to denote
a lower bound that is not asymptotically tight. One way
to define it is by
We formally define ω(g(n)) (“little-omega of g of n”) as
the set
ω(g(n)) = { f(n) : for every positive constants c, there
exist a positive constant n0 such that 0 ≤ c g(n) < f(n) for
all n ≥ n0 }
For example, n2/ 3 = ω(n) but n2/ 3 ≠ ω(n2)

17. Asymptotic Notations
Example: Let f(n) = 2n2 + 3n + 5
f(n) = 2n2 + 3n + 5 > c n --------(2)
For every positive constant c there exist a positive
constant n0 such that eqn (2) is true.
For smaller values of c, eqn (2) is obviously true.
For larger values of c for ex 1000
2n2 + 3n + 5 > 1000 n for every n ≥ 1000
As f(n) > cg(n), where c = 1000 and g(n) = n
for all n ≥ 1000
So f(n) = 2n2 + 3n + 5 = ω( n )

18. Asymptotic Notations
From all five notations we can conclude the following:
f(n) = θ(g(n)) <=> f(n) = O(g(n)) AND f(n) = Ω(g(n))
f(n) = o(g(n)) <=> f(n) = O(g(n)) AND f(n) ≠ Ω(g(n))
f(n) = ω(g(n)) <=> f(n) ≠ O(g(n)) AND f(n) = Ω(g(n))
f(n) = o(g(n)) => f(n) ≠ ω(g(n))
f(n) = ω(g(n)) => f(n) ≠ o(g(n))

19. Asymptotic Notations
To Prove asymptotic bounds on any given functions f(n)
and g(n) we can use following mathematical definitions:
constant => f(n) = θ(g(n))
0 => f(n) = o(g(n))
∞ => f(n) = ω(g(n))

20. Asymptotic Notations
Examples:
1. Prove that 2n2 + 5n +2 = θ(n2)
limn→ ∞ f(n) / g(n) = limn→ ∞ (2n2 + 5n +2) / (n2)
= limn→ ∞ (2 + 5/n + 2/n2)
= 2 (Constant)
so 2n2 + 5n +2 = θ(n2)
2. Prove or disprove that 5n3+ 3n +1 = o(n2)
limn→ ∞ f(n) / g(n) = limn→ ∞ (5n3 + 3n +1) / (n2)
= limn→ ∞ (5n + 3/n + 1/n2)
= ∞
so 5n3+ 3n +1 = ω(n2)
and so 5n3+ 3n +1 ≠ o(n2)

21. Asymptotic Notations
Examples:
3. Prove that (n + a)b = θ(nb)
limn→ ∞ f(n) / g(n) = limn→ ∞ (n + a)b / (nb)
= limn→ ∞ nb(1 + a/n)b / (nb)
= limn→ ∞ (1 + a/n)b
= 1b = 1
so (n + a)b = θ(nb)
4. Prove that 2n+1 = O(2n)
limn→ ∞ f(n) / g(n) = limn→ ∞ (2n+1 / 2n)
= 2
so 2n+1 = θ(2n)
so 2n+1 = O(2n)

22. Asymptotic Notations
Examples: Second Method
3. Prove that (n + a)b = θ(nb)
f(n) = (n + a)b
= bC0nba0 + bC1nb-1a1 + bC2nb-2a2 ……+ bCbn0ab
≤ ( bC0a0 + bC1a1 + bC2a2 ……+ bCbab ) nb
= D. nb
= O(nb)………(1)
Now f(n) = (n + a)b
= bC0nba0 + bC1nb-1a1 + bC2nb-2a2 ……+ bCbn0ab
≥ bC0a0 . nb
= K. nb
= Ω(nb)…....(2)
From (1) and (2) (n + a)b = θ(nb)

23. Asymptotic Notations
4. Prove that 2n+1 = O(2n)
2n+1 = 2 . 2n
≤ 3 . 2n
so 2n+1 = O(2n)
5. Prove that max( f(n) , g(n) ) = θ (f(n) + g(n) )
Let F(n) = max( f(n) , g(n) )
and G(n)= (f(n) + g(n) )
So we need to prove F(n) = θ (G(n))

24. Asymptotic Notations
For Upper bound
max( f(n) , g(n) ) ≤ 1. (f(n) + g(n) ) for all n ≥ 1
So max( f(n) , g(n) ) = O (f(n) + g(n) ) …… (1)

25. Asymptotic Notations
For Upper bound
max( f(n) , g(n) ) ≤ 1. (f(n) + g(n) ) for all n ≥ 1
So max( f(n) , g(n) ) = O (f(n) + g(n) ) …… (1)
For Lower bound
max( f(n) , g(n) ) ≥ 1/2. (f(n) + g(n) ) for all n ≥ 1
So max( f(n) , g(n) ) = Ω (f(n) + g(n) ) ……. (2)
From (n) and (2)
max( f(n) , g(n) ) = θ (f(n) + g(n) )

26. THANK YOU