Asymptotic notations

1. Complexity Analysis
Presented
By:

2. Comparing Algorithms
• Given 2 or more algorithms to solve the same problem, how do we select the best one?
• Some criteria for selecting an algorithm
1) Is it easy to implement, understand, modify?
2) How long does it take to run it to completion?
3) How much of computer memory does it use?
• Software engineering is primarily concerned with the first criteria
• Time complexity
• The amount of time that an algorithm needs to run to completion
• Space complexity
• The amount of memory an algorithm needs to run

3. How to Calculate Running Time
• Best case running time is usually
useless
• Average case time is very useful
but often difficult to determine
• We focus on the worst case
running time
• Easier to analyze
• Crucial to applications such as
games, finance and robotics
0
20
40
60
80
100
120
RunningTime
1000 2000 3000 4000
I nput Size
best case
average case
worst case

4. Asymptotic Algorithm Analysis
• The asymptotic analysis of an algorithm determines the running
time in big-Oh notation
• To perform the asymptotic analysis
• We find the worst-case number of primitive operations executed as a
function of the input size
• We express this function with big-Oh notation
• Example:
• We determine that algorithm arrayMax executes at most 7n − 1 primitive
operations
• We say that algorithm arrayMax “runs in O(n) time”
• Since constant factors and lower-order terms are eventually
dropped anyhow, we can disregard them when counting primitive
operations

5. Big-Oh Notation
• asymptotic analysis
• The big-Oh notation is used widely to characterize
running times and space bounds
• The big-Oh notation allows us to ignore constant factors
and lower order terms and focus on the main components
of a function which affect its growth
• Given functions f(n) and g(n), we say that f(n) is O(g(n)) if
there are positive constants
c and n0 such that
f(n) ≤ cg(n) for n ≥ n0

6. Big-Oh Notation Example
Example: 2n + 10 is
O(n)
 2n + 10 ≤ cn
 (c − 2) n ≥ 10
 n ≥ 10/(c − 2)
 Pick c = 3 and n0= 10
0
10
20
30
40
50
60
70
80
0 5 10 15 20 25 30
n
3n
2n+10
n

7. Big-Oh and Growth Rate
• The big-Oh notation gives an upper bound on the growth rate of a
function
• The statement “f(n) is O(g(n))” means that the growth rate of f(n)
is no more than the growth rate of g(n)
• We can use the big-Oh notation to rank functions according to
their growth rate
f(n) is O(g(n)) g(n) is O(f(n))
g(n) grows more Yes No
f(n) grows more No Yes
Same growth Yes Yes

8. Relatives of Big-Oh
big-Omega
f(n) is Ω(g(n)) if there is a constant c > 0
and an integer constant n0 ≥ 1
such that
f(n) ≥ c•g(n)
for n ≥ n0
big-Theta
f(n) is Θ(g(n)) if there are constants c’ > 0 and c’’ > 0 and an integer constant n0 ≥ 1
such that c’•g(n) ≤ f(n) ≤ c’’•g(n)
 for n ≥ n0

9. Important Functions Growth Rates
n log(n) n nlog(n) n2
n3
2n
8 3 8 24 64 512 256
16 4 16 64 256 4096 65536
32 5 32 160 1024 32768 4.3x109
64 6 64 384 4096 262144 1.8x1019
128 7 128 896 16384 2097152 3.4x1038
256 8 256 2048 65536 1677721
8
1.2x1077

10. More Big-Oh Examples
7n-2
7n-2 is O(n)
need c > 0 and n0 ≥ 1 such that 7n-2 ≤ c•n for n ≥ n0
this is true for c = 7 and n0 = 1

3n3
+ 20n2
+ 5
3n3
+ 20n2
+ 5 is O(n3
)
need c > 0 and n0 ≥ 1 such that 3n3
+ 20n2
+ 5 ≤ c•n3
for n ≥ n0
this is true for c = 4 and n0 = 21

3 log n + 5
3 log n + 5 is O(log n)
need c > 0 and n0 ≥ 1 such that 3 log n + 5 ≤ c•log n for n ≥ n0
this is true for c = 8 and n0 = 2