Enzyme, Pharmaceutical Aids, Miscellaneous Last Part of Chapter no 5th.pdf
Chapter two
1. Computer Science and Engineering Program
Fundamental of the Analysis of Alg.
Efficiency
Gedamu A.
alemugedamu@yahoo.com
Image processing and Computer vision SIG
Some of the sides are exported from different sources to clarify the topic
2. Computer Science and Engineering Program
Analysis of algorithms
Issues:
Correctness
space efficiency
time efficiency
optimality
Approaches:
theoretical analysis
empirical analysis
3. Computer Science and Engineering Program
Space Analysis
When considering space complexity, algorithms are divided
into those that need extra space to do their work and those that
work in place.
Space analysis would examine all of the data being stored to
see if there were more efficient ways to store it.
Example : As a developer, how do you store the real numbers ?
Suppose we are storing a real number that has only one place of precision
after the decimal point and ranges between -10 and +10.
How many bytes you need ?
4. Computer Science and Engineering Program
Space Analysis
Example : As a developer, how do you store the real numbers ?
Suppose we are storing a real number that has only one place of precision
after the decimal point and ranges between -10 and +10.
How many bytes you need ?
Most computers will use between 4 and 8 bytes of memory.
If we first multiply the value by 10. We can then store this as an integer
between -100 and +100. This needs only 1 byte, a savings of 3 to 7 bytes.
A program that stores 1000 of these values can save 3000 to 7000 bytes.
It makes a big difference when programming mobile or PDAs or when you
have large input .
5. Computer Science and Engineering Program
Theoretical analysis of time efficiency
Time efficiency is analyzed by determining the number of repetitions of the
basic operation as a function of input size
Basic operation: the operation that contributes the most towards the
running time of the algorithm
T(n) ≈ copC(n)
running time execution time
for basic operation
or cost
Number of times
basic operation
is executed
input size
Note: Different basic operations may cost differently!
6. Computer Science and Engineering Program
Why Input Classes are Important?
Input determines what the path of execution through an
algorithm will be.
If we are interested in finding the largest value in a list of N
numbers, we can use the following algorithm:
7. Computer Science and Engineering Program
Why Input Classes are Important?
If the list is in decreasing order,
There will only be one assignment done before the loop starts.
If the list is in increasing order,
There will be N assignments (one before the loop starts and N -1 inside the
loop).
Our analysis must consider more than one possible set of input,
because if we only look at one set of input, it may be the set that is
solved the fastest (or slowest).
8. Computer Science and Engineering Program
Input size and basic operation examples
Problem Input size measure Basic operation
Searching for key in a list
of n items
Number of list’s items, i.e. n Key comparison
Multiplication of two
matrices
Matrix dimensions or total
number of elements
Multiplication of two
numbers
Checking primality of a
given integer n
n’size = number of digits (in
binary representation)
Division
Typical graph problem #vertices and/or edges
Visiting a vertex or
traversing an edge
9. Computer Science and Engineering Program
Importance of the analysis
It gives an idea about how fast the algorithm
If the number of basic operations
C(n) = ½ n (n-1) = ½ n2 – ½ n ≈ ½ n2
How much longer if the algorithm doubles its input
size?
Increasing input size increases the complexity
We tend to omit the constants because they have no effect with large inputs
Everything is based on estimation
T(n) ≈ copC(n)
4
)(
2
1
)2(
2
1
)(
)2(
2
2
)(
)2(
n
n
nCC
nCC
OP
OP
nT
nT
10. Computer Science and Engineering Program
Empirical analysis of time efficiency
Select a specific (typical) sample of inputs
Use physical unit of time (e.g., milliseconds)
or
Count actual number of basic operation’s executions
Analyze the empirical/experimental data
11. Computer Science and Engineering Program
case to consider in analysis
Best-case, average-case, worst-case
For some algorithms, efficiency depends on form of input:
Worst case: Cworst(n) – maximum over inputs of size n
Best case: Cbest(n) – minimum over inputs of size n
Average case: Cavg(n) – “average” over inputs of size n
The toughest to do
12. Computer Science and Engineering Program
Best-case, average-case, worst-case
Average case: Cavg(n) – “average” over inputs of size n
Determine the number of different groups into which all possible input
sets can be divided.
Determine the probability that the input will come from each of these
groups.
Determine how long the algorithm will run for each of these groups.
n is the size of the input,
m is the number of groups,
pi is the probability that the input will be from
group i,
ti is the time that the algorithm takes for input
from group i.
13. Computer Science and Engineering Program
Example: Sequential search
Worst case
Best case
Average case
n key comparisons
1 comparison
(n+1)/2, assuming K is in A
14. Computer Science and Engineering Program
Computing the Average Case for the Sequential
search
Neither the Worst nor the Best case gives the yield to the actual performance of an algorithm
with random input.
The Average Case does
Assume that:
The probability of successful search is equal to p(0≤ p ≤1)
The probability of the first match occurring in the ith position is the same for every i .
The probability of a match occurs at ith position is p/n for every i
In the case of unsuccessful search , the number of comparison is n with probability (1-
p).
)1(
2
)1(
)1.(
2
)1(
.
)1.(]...321[
)1.(]....3.2.1[)(
pn
np
pn
nn
n
p
pnn
n
p
pn
n
p
n
n
p
n
p
n
p
nCavg
15. Computer Science and Engineering Program
Computing the Average Case for the Sequential
search
If p =1 (I found the key k)
The average number of comparisons is (n+1)/2
If p=0
The average number of key comparisons is n
The average Case is more difficult than the Best and Worst cases
)1(
2
)1(
)1.(
2
)1(
.
)1.(]...321[
)1.(]....3.2.1[)(
pn
np
pn
nn
n
p
pnn
n
p
pn
n
p
n
n
p
n
p
n
p
nCavg
18. Computer Science and Engineering Program
Logarithms
Which Base ?
Loga n = Loga b Logb n
Loga n = c Logb n
In terms of complexity , we tend to
ignore the constant
22. Computer Science and Engineering Program
Types of formulas for basic operation’s count
Exact formula
e.g., C(n) = n(n-1)/2
Formula indicating order of growth with specific multiplicative
constant
e.g., C(n) ≈ 0.5 n2
Formula indicating order of growth with unknown
multiplicative constant
e.g., C(n) ≈ cn2
24. Computer Science and Engineering Program
Order of growth
Of greater concern is the rate of increase in operations for an
algorithm to solve a problem as the size of the problem
increases.
This is referred to as the rate of growth of the algorithm.
25. Computer Science and Engineering Program
Order of growth
The function based on x2 increases slowly at first, but as the problem size
gets larger, it begins to grow at a rapid rate.
The functions that are based on x both grow at a steady rate for the entire
length of the graph.
The function based on log x seems to not grow at all, but this is because it
is actually growing at a very slow rate.
26. Computer Science and Engineering Program
Values of some important functions as n
27. Computer Science and Engineering Program
Order of growth
Second point to consider :
Because the faster growing functions increase at such a
significant rate, they quickly dominate the slower-growing
functions.
This means that if we determine that an algorithm’s
complexity is a combination of two of these classes, we will
frequently ignore all but the fastest growing of these terms.
Example : if the complexity is
we tend to ignore 30x term
28. Computer Science and Engineering Program
Classification of Growth
Asymptotic order of growth
A way of comparing functions that ignores constant factors and small
input sizes.
O(g(n)): class of functions f(n) that grow no faster than g(n)
All functions with smaller or the same order of growth as g(n)
Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)
All functions that are larger or have the same order of growth as g(n)
Θ(g(n)): class of functions f(n) that grow at same rate as g(n)
Set of functions that have the same order of growth as g(n)
)(),()1(5.0),(5100),( 23222
nOnnOnnnOnnOn
),(5100),()1(5.0),( 2223
nnnnnnn
)( 22
nbnan
29. Computer Science and Engineering Program
Big-oh
•O(g(n)): class of functions t(n) that grow no faster than g(n)
• if there exist some positive constant c and some nonnegative n0
such that
0)()( nnallforncgnt
)O(n5100nthatProve:Ex 2
5101
101n101n
5)nall(forn100n5100n
0
2
nandc
You may come up with different c and n0
30. Computer Science and Engineering Program
Big-omega
Ω(g(n)): class of functions t(n) that grow at least as fast as g(n)
?)(
)()(
23
0
nnthatprove
nnallforncgnt
01
0
0
23
nandc
nallfornn
31. Computer Science and Engineering Program
Big-theta
Θ(g(n)): class of functions t(n) that grow at same rate as g(n)
02 )(1)()( nnallforngcntngc
021 ,, nandcc
gettoneedYou
32. Computer Science and Engineering Program
(g(n)), functions that grow at least as fast as g(n)
(g(n)), functions that grow at the same rate as g(n)
O(g(n)), functions that grow no faster than g(n)
g(n)
>=
<=
=
Summary
33. Computer Science and Engineering Program
Theorem
If t1(n) O(g1(n)) and t2(n) O(g2(n)), then
t1(n) + t2(n) O(max{g1(n), g2(n)}).
The analogous assertions are true for the -notation and -
notation.
Implication: The algorithm’s overall efficiency will be
determined by the part with a larger order of growth, i.e., its
least efficient part.
For example, 5n2 + 3nlogn O(n2)
Proof. There exist constants c1, c2, n1, n2 such that
t1(n) c1*g1(n), for all n n1
t2(n) c2*g2(n), for all n n2
Define c3 = c1 + c2 and n3 = max{n1,n2}. Then
t1(n) + t2(n) c3*max{g1(n), g2(n)}, for all n n3
34. Computer Science and Engineering Program
Some properties of asymptotic order of growth
f(n) O(f(n))
f(n) O(g(n)) iff g(n) (f(n))
If f (n) O(g (n)) and g(n) O(h(n)) , then f(n) O(h(n))
If f1(n) O(g1(n)) and f2(n) O(g2(n)) , then
f1(n) + f2(n) O(max{g1(n), g2(n)})
Also, 1in (f(i)) = (1in f(i))
35. Computer Science and Engineering Program
Orders of growth of some important functions
All logarithmic functions loga n belong to the same class
(log n) no matter what the logarithm’s base a > 1 is
because
All polynomials of the same degree k belong to the same class:
aknk + ak-1nk-1 + … + a0 (nk)
Exponential functions an have different orders of growth for
different a’s
ann bba log/loglog
36. Computer Science and Engineering Program
Basic asymptotic efficiency classes
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial
37. Computer Science and Engineering Program
Time efficiency of nonrecursive algorithms
General Plan for Analysis
Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best cases for input of size n
Set up a sum for the number of times the basic operation is executed
Simplify the sum using standard formulas and rules.
38. Computer Science and Engineering Program
Example 1: The Largest Element
T(n) = 1in-1 (1) = n-1 = (n) comparisons
39. Computer Science and Engineering Program
Example 2: Element uniqueness problem
T(n) = 0in-2 (i+1jn-1 (1))
= 0in-2 (n-i+1) =
= ( ) comparisons2
n
)(
2
1
2
)1(
1.....)2()1(1
11
22
2
0
nn
nn
nnin
lu
n
i
u
li
40. Computer Science and Engineering Program
Example 3: Matrix multiplication
T(n) = = ( ) multiplications
Ignored addition for simplicity
3
n
1
0
1
0
1
0
1
n
i
n
j
n
k
41. Computer Science and Engineering Program
Example 4: Counting binary digits
It cannot be investigated the way the previous examples are.
The halving game: Find integer i such that n/ ≤ 1.
Answer: i ≤ log n. So, T(n) = (log n) divisions.
Another solution: Using recurrence relations.
i
2
42. Computer Science and Engineering Program
Plan for Analysis of Recursive Algorithms
Decide on a parameter indicating an input’s size.
Identify the algorithm’s basic operation.
Check whether the number of times the basic op. is executed may
vary on different inputs of the same size. (If it may, the worst,
average, and best cases must be investigated separately.)
Set up a recurrence relation with an appropriate initial condition
expressing the number of times the basic op. is executed.
Solve the recurrence (or, at the very least, establish its solution’s
order of growth) by backward substitutions or another method.
43. Computer Science and Engineering Program
Example 1 : Factorial: Recursive Algorithm
43
The stopping condition is n=0
A repetitive algorithm uses recursion whenever the algorithm
appears within the definition itself.
44. Computer Science and Engineering Program
Example 1: Recursive evaluation of n!
Definition: n ! = 1 2 … (n-1) n for n ≥ 1 and 0! = 1
Recursive definition of n!: F(n) = F(n-1) n for n ≥ 1 and
F(0) = 1
Size:
Basic operation:
Recurrence relation:
n
multiplication
M(n) = M(n-1) (for F(n-1) ) + 1 (for the n * F(n-1))
M(0) = 0
45. Computer Science and Engineering Program
Solving the recurrence for M(n)
M(n) = M(n-1) + 1, M(0) = 0 (no multiplication when n=0)
M(n) = M(n-1) + 1
= (M(n-2) + 1) + 1 = M(n-2) + 2
= (M(n-3) + 1) + 2 = M(n-3) + 3
…
= M(n-i) + i
= M(0) + n
= n
The method is called backward substitution.
46. Computer Science and Engineering Program
Example 2: The Tower of Hanoi Puzzle
46
Towers of Hanoi (aka Tower of Hanoi) is a mathematical puzzle
invented by a French Mathematician Edouard Lucas in 1983.
•Initially the game has few discs arranged in the increasing
order of size in one of the tower.
• The number of discs can vary, but there are only three towers.
•The goal is to transfer the discs from one tower another tower.
However you can move only one disk at a time and you can
never place a bigger disc over a smaller disk. It is also
understood that you can only take the top most disc from any
given tower.
47. Computer Science and Engineering Program
Example 2: The Tower of Hanoi Puzzle
Recurrence for number of moves: M(n) = 2M(n-1) + 1
48. Computer Science and Engineering Program
1
2
3
4
5
6
7
0 Move Sequence for d = 3
49. Computer Science and Engineering Program
The Tower of Hanoi Puzzle
Here's how to find the number of moves needed to transfer larger
numbers of disks from post A to post C, when M = the number of
moves needed to transfer n-1 disks from post A to post C:
for 1 disk it takes 1 move to transfer 1 disk from post A to post C;
for 2 disks, it will take 3 moves: 2M + 1 = 2(1) + 1 = 3
for 3 disks, it will take 7 moves: 2M + 1 = 2(3) + 1 = 7
for 4 disks, it will take 15 moves: 2M + 1 = 2(7) + 1 = 15
for 5 disks, it will take 31 moves: 2M + 1 = 2(15) + 1 = 31
for 6 disks... ?
50. Computer Science and Engineering Program
Explicit Pattern
Number of Disks Number of Moves
1 1
2 3
3 7
4 15
5 31
6 63
51. Computer Science and Engineering Program
Powers of two help reveal the pattern:
Number of Disks (n) Number of Moves
1 21 - 1 = 2 - 1 = 1
2 22 - 1 = 4 - 1 = 3
3 23 - 1 = 8 - 1 = 7
4 24 - 1 = 16 - 1 = 15
5 25 - 1 = 32 - 1 = 31
6 26 - 1 = 64 - 1 = 63
52. Computer Science and Engineering Program
Fascinating fact
So the formula for finding the number of steps it takes to transfer
n disks from post A to post C is:
2
n - 1