Data Structure and Algorithm Chapter One Lecture Note

1. Chapter One
Algorithm Analysis Concepts
12/13/2022 1

2. Algorithm Analysis Concepts
1. Measuring Complexity
 Time complexity
 Space complexity
2. Complexity of Algorithms
3. Big-Oh Notation and others
12/13/2022 2

3. Measuring Complexity
• An algorithm's space and time complexity can be
used to determine its effectiveness.
• To determine the efficacy of a program or
algorithm, understanding how to evaluate them
using Space and Time complexity can help the
program perform optimally under specified
conditions.
• As a result, you become more efficient
programmer
• "Measurements" are machine independent
– worst-, average-, best-case analysis
12/13/2022 3

4. … cont
• To express the efficiency of our algorithms which
of the three notations should we use?
• As computer scientist we generally like to express
our algorithms as big O since we would like to
know the upper bounds of our algorithms.
• Why?
• If we know the worse case then we can aim to
improve it and/or avoid it.
12/13/2022 4

5. Complexity of Algorithms
Standard Analysis Techniques
• Constant time statements
• Analyzing Loops
• Analyzing Nested Loops
• Analyzing Sequence of Statements
• Analyzing Conditional Statements
12/13/2022 5

6. Complexity Analysis
• Complexity Analysis is the systematic study of the cost of
computation, measured either in time units or in operations
performed, or in the amount of storage space required.
• There are two things to consider:
• Time Complexity: Determine the approximate number of operations
required to solve a problem of size n.
• Space Complexity: Determine the approximate memory required to
solve a problem of size n.
• Complexity analysis involves two distinct phases:
• Algorithm Analysis: Analysis of the algorithm or data structure to
produce a function T (n) that describes the algorithm in terms of the
operations performed in order to measure the complexity of the
algorithm.
• Order of Magnitude Analysis: Analysis of the function T (n) to
determine the general complexity category to which it belongs
12/13/2022 6

7. … cont
Examples: int x=10; x=x+10; cout<< x;
Time Units to Compute:
• 1 for the assignment statement: int x=10;
• 1 for the single arithmetic statement: x +10
• 1 for the assignment statement: x=x+10;
• 1 for the output statement: cout<< x;
T (n)= 1+1+1+1 = 4
12/13/2022 7

8. Constant time statements
• Simplest case: O(1) time statements
• Assignment statements of simple data types
int x = y;
• Arithmetic operations:
x = 5 * y + 4 - z;
• Array referencing:
A[j] = 5;
• Array assignment:
 j, A[j] = 5;
• Most conditional tests:
if (x < 12) ...
8
12/13/2022

9. Analyzing Loops[1]
• Time Units to Compute: 1 for the initialization expression: i=1 ; n+1 for
the test expression: i<= n; n for the increment expression: i++ ; n for the
output statement: cout<<i;
• T (n)= 1+ n+1+n+n = 3n +2
• Any loop has two parts:
– How many iterations are performed?
– How many steps per iteration?
int sum = 0,j;
for (j=0; j < N; j++)
sum = sum +j;
– Loop executes N times (0..N-1)
– 4 = O(1) steps per iteration
• Total time is N * O(1) = O(N*1) = O(N)
9
12/13/2022

10. Analyzing Loops[2]
• What about this for loop?
int sum =0, j;
for (j=0; j < 100; j++)
sum = sum +j;
• Loop executes 100 times
• 4 = O(1) steps per iteration
• Total time is 100 * O(1) = O(100 * 1) = O(100) = O(1)
10
12/13/2022

11. Analyzing Loops – Linear Loops
• Example (have a look at this code segment):
• Efficiency is proportional to the number of iterations.
• Efficiency time function is :
f(n) = 1 + (n-1) + c*(n-1) +( n-1)
= (c+2)*(n-1) + 1
= (c+2)n – (c+2) +1
• Asymptotically, efficiency is : O(n)
11
12/13/2022

12. Analyzing Nested Loops[1]
• Treat just like a single loop and evaluate each
level of nesting as needed:
int j,k;
for (j=0; j<N; j++)
for (k=N; k>0; k--)
sum += k+j;
• Start with outer loop:
– How many iterations? N
– How much time per iteration? Need to evaluate inner loop
• Inner loop uses O(N) time
• Total time is N * O(N) = O(N*N) = O(N2)
12
12/13/2022

13. Analyzing Nested Loops[2]
• What if the number of iterations of one loop
depends on the counter of the other?
int j,k;
for (j=0; j < N; j++)
for (k=0; k < j; k++)
sum += k+j;
• Analyze inner and outer loop together:
• Number of iterations of the inner loop is:
• 0 + 1 + 2 + ... + (N-1) = O(N2)
13
12/13/2022

14. How Did We Get This Answer?
• When doing Big-O analysis, we sometimes have
to compute a series like: 1 + 2 + 3 + ... + (n-1) + n
• i.e. Sum of first n numbers. What is the
complexity of this?
• Gauss figured out that the sum of the first n
numbers is always:
14
12/13/2022

15. Sequence of Statements
• For a sequence of statements, compute their
complexity functions individually and add them
up
• Total cost is O(n2) + O(n) +O(1) = O(n2)
15
12/13/2022

16. Deriving A Recurrence Equation
• So far, all algorithms that we have been analyzing
have been non recursive
• Example : Recursive power method
• If N = 1, then running time T(N) is 2
• However if N ≥ 2, then running time T(N) is the cost of each step taken plus time
required to compute power(x,n-1). (i.e. T(N) = 2+T(N-1) for N ≥ 2)
• How do we solve this? One way is to use the iteration method.
16
12/13/2022

17. Iteration Method
• This is sometimes known as “Back Substituting”.
• Involves expanding the recurrence in order to see a
pattern.
• Solving formula from previous example using the
iteration method :
• Solution : Expand and apply to itself :
Let T(1) = n0 = 2
T(N) = 2 + T(N-1)
= 2 + 2 + T(N-2)
= 2 + 2 + 2 + T(N-3)
= 2 + 2 + 2 + ……+ 2 + T(1)
= 2N + 2 remember that T(1) = n0 = 2 for N = 1
• So T(N) = 2N+2 is O(N) for last example.
17
12/13/2022

18. Big-Oh Notation and others
• Big-Oh notation is a way of comparing algorithms and is
used for computing the complexity of algorithms; i.e., the
amount of time that it takes for computer program to run.
• In theoretical terms, Big – O notation is used to examine
the performance/complexity of an algorithm.
• Big – O notation examines an algorithm's upper bound of
performance, i.e. its worst-case behavior.
• Big –O notation also considers asymptotic algorithm
behavior, which refers to the method's performance when
the amount of the input climbs to extremely big.
• Computational complexity asymptote O(f) measures the
order of employed resources based on the magnitude of
the input data (CPU time, RAM, etc.).
12/13/2022 18

19. 12/13/2022 19

20. … cont
O(n)
12/13/2022 20

21. … cont
O(n^2)
12/13/2022 21

22. … cont
(n(n+1)/2)
O(n^2)
12/13/2022 22

23. … cont
12/13/2022 23

24. … cont
12/13/2022 24

25. … cont
12/13/2022 25

26. … cont
12/13/2022 26

27. … cont
12/13/2022 27

28. … cont
• Asymptotically less than or equal to O
• Asymptotically greater than or equal to 
• Asymptotically equal to 
• Asymptotically strictly less o
12/13/2022 28

29. … cont
• Other commonly used notations
–Big Oh Notation: Upper bound
–Omega Notation: Lower bound
–Theta Notation: Tighter bound
12/13/2022 29

30. Thank You!
?
12/13/2022 30