Asymptotic Notaton

1. Advanced Algorithms
(PCCO6020T)
Unit 1
Prof. Swapnil H. Chaudhari
Computer Engineering Department,
R.C.Patel Institute of Technology, Shirpur

2. Syllabus Structure

9. Introduction
Definition of algorithms:
 A set of instructions that solve a specific problem or accomplish a specific
task.
Importance of algorithms in computer science:
 Algorithms form the backbone of computer science and are essential in
fields such as artificial intelligence, data science, cryptography, and more.
Types of algorithms:
 There are many different types of algorithms, including search algorithms,
sorting algorithms, graph algorithms, and dynamic programming
algorithms, among others.

10. Introduction
Time and space complexity:
 Two important factors in evaluating the efficiency of an algorithm are its
time complexity, which refers to the amount of time it takes to run, and its
space complexity, which refers to the amount of memory it requires.
 Big O notation: Big O notation is a mathematical notation used to describe
the upper bound of an algorithm's time complexity. It provides a way to
express the worst-case scenario for an algorithm's running time.

11. What is time complexity
 Time complexity is a function that describes the amount of time required
to run an algorithm, as input size of the algorithm
 Calculating time complexity by the growth of the algorithm is the most
reliable way of calculating the efficiency of an algorithm

12. Calculate the Time Complexity
 A common mistake with time complexity is to think of it as the running
time(clock time) of an algorithm.
 The running time of the algorithm is how long it takes the computer to
execute the lines of code to completion, usually measured in milliseconds or
seconds.
 Using this method is not the most efficient way of calculating the running
time of an algorithm, cause the running time of an algorithm depends on
 The speed of the computer (Hardware).
 The programming language used (Java, C++, Python).
 The compiler that translates our code into machine code (Clang, GCC, Min
GW).

13.  Consider a model machine
 Assigning values to variables.
 Making comparisons.
 Executing arithmetic operations.
 Accessing objects from memory.

14.  Time Complexity:
 In the above code “Hello World” is printed only once on the screen.
So, the time complexity is constant: O(1)

15.  1) Loop
for( i = 1 ; i < = n ; i++)
{
x= y + z;
}

16. Amortized Analysis
16

17. Amortized Analysis
 Amortized analysis is applied on data structures that
support many operations.
 The sequence of operations and the multiplicity of each
operation is application specific or the associated
algorithm specific.
 Classical asymptotic analysis gives worst case analysis of
each operation without taking the effect of one operation
on the other.
 Amortized analysis focuses on a sequence of operations,
an interplay between operations, and thus yielding an
analysis which is precise and depicts a micro-level
analysis.
17

18. Amortized Analysis
 Purpose is to accurately compute the total time spent in
executing a sequence of operations on a data structure
 Three different approaches:
Aggregate method
Accounting method
Potential method
18

19. Aggregate method
 We determine an upper bound T(n) on Total cost of a sequence of n-operations.
 In the worst case :
 The average cost or amortized cost per operation is =
𝑇(𝑛)
𝑛
 Note that this amortized cost applies to each operation, even when there is several
types of operations in sequence.
Note that this amortized cost applies to each operation, even when there is several
types of operations in sequence.
19

20. How large should a hash
table be?
Goal: Make the table as small as possible, but
large enough so that it won’t overflow (or
otherwise become inefficient).
Problem: What if we don’t know the proper size
in advance?
Solution: Dynamic tables.
IDEA: Whenever the table overflows, “grow” it
by allocating (via malloc or new) a new, larger
table. Move all items from the old table into the
new one, and free the storage for the old table.

21. Example of a dynamic table
1. INSERT
2. INSERT
1
overflow

22. 1
Example of a dynamic table
1. INSERT
2. INSERT overflow

23. 1
2
Example of a dynamic table
1. INSERT
2. INSERT

24. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
1
2
overflow

25. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
2
1
overflow

26. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
2
1

27. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
4. INSERT 4
3
2
1

28. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
4. INSERT
5. INSERT
4
3
2
1
overflow

29. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
4. INSERT
5. INSERT
4
3
2
1
overflow

30. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
4. INSERT
5. INSERT
4
3
2
1

31. Example of a dynamic table
1. INSERT
2. INSERT
3. INSERT
4. INSERT
5. INSERT
6. INSERT
7. INSERT
6
5
4
3
2
1
7

32. Worst-case analysis
Consider a sequence of n insertions. The worst-case time
to execute one insertion is (n).Therefore, the worst-case
time for n insertions is n · (n) = (n2).
WRONG! In fact, the worst-case cost for
n insertions is only (n) ≪ (n2).
Let’s see why.

33. Tighter analysis
i 1 2 3 4 5 6 7 8 9 10
sizei 1 2 4 4 8 8 8 8 16 16
ci 1 2 3 1 5 1 1 1 9 1
Let ci = the cost of the ith insertion
=
i if i – 1 is an exact power of 2,
1 otherwise.

34. Tighter analysis
Let ci = the cost of the ith insertion
=
i if i – 1 is an exact power of 2,
1 otherwise.
i 1 2 3 4 5 6 7 8 9 10
sizei 1 2 4 4 8 8 8 8 16 16
ci
1 1
1
1
2
1 1
4
1 1 1 1
8
1

35. Tighter analysis (continued)
2 j
 n 
n
lg(n1)

j0
Cost of n insertions  ci
i1
 3n
 (n).
Thus, the average cost of each dynamic-table
operation is (n)/n = (1).

36. Example for amortized analysis
• Amortized analysis can be used to show that the average cost of an operation is
small , if one averages over a sequence of operations ,even though a single
operation within the sequence might be expensive
• Stack operations:
– PUSH(S,x), O(1)
– POP(S), O(1)
– MULTIPOP(S,k), min(s,k)
•while not STACK-EMPTY(S) and k>0
• do POP(S)
• k=k-1
• Let us consider a sequence of n PUSH, POP, MULTIPOP.
– The worst case cost for MULTIPOP in the sequence is O(n), since the stack
size is at most n.
– thus the cost of the sequence is O(n2). Correct, but not tight.

37. Aggregate Analysis
• In fact, a sequence of n operations on an
initially empty stack cost at most O(n). Why?
Each object can be POP only once (including in MULTIPOP) for each time
it is PUSHed. #POPs is at most #PUSHs, which is at most n.
Thus the average cost of an operation is O(n)/n = O(1).
Amortized cost in aggregate analysis is defined to be average cost.

38. Another example: increasing a binary counter
• Binary counter of length k, A[0..k-1] of bit array.
• INCREMENT(A)
1. i0
2. while i<k and A[i]=1
3. do A[i]0 (flip, reset)
4. ii+1
5. if i<k
6. then A[i]1 (flip, set)

39. Amortized (Aggregate) Analysis of INCREMENT(A)
Observation: The running time determined by #flips but not all bits flip each time INCREMENT is
called.
A[0] flips every time, total n times.
A[1] flips every other time, n/2 times.
A[2] flips every forth time, n/4 times.
….
for i=0,1,…,k-1, A[i] flips n/2i times.
Thus total #flips is
=2n.
𝑖=0
log 𝑛
𝑛
2𝑖 <
𝑖=0
∞
𝑛
2𝑖

40. Accounting Method
 In accounting method, we assign different charges to different
operations. The amount we charge is called amortized cost
 𝐶𝑖 = is the actual cost
 𝐶𝑖 =is the amortized cost

41. Accounting method
• Charge ith operation a fictitious amortized cost
ĉi, where $1 pays for 1 unit of work (i.e., time).
• This fee is consumed to perform the operation.
• Any amount not immediately consumed is stored
in the bank for use by subsequent operations.
• The bank balance must not go negative! We
must ensure that n n
i1 i1
ci  cˆi
for all n.
• Thus, the total amortized costs provide an upper
bound on the total true costs.

42. Charge an amortized cost of ĉi = $3 for the ith
insertion.
• $1 pays for the immediate insertion.
• $2 is stored for later table doubling.
When the table doubles, $1 pays to move a recent
item, and $1 pays to move an old item.
Accounting analysis of
dynamic tables

43. Accounting analysis
(continued)
Key invariant: Bank balance never drops below 0.
Thus, the sum of the amortized costs provides an
upper bound on the sum of the true costs.
i 1 2 3 4 5 6 7 8 9 10
sizei 1 2 4 4 8 8 8 8 16 16
ci 1 2 3 1 5 1 1 1 9 1
ĉi 2* 3 3 3 3 3 3 3 3 3
banki 1 2 2 4 2 4 6 8 2 4

46. Potential method
IDEA: View the bank account as the potential
energy (à la physics) of the dynamic set.
Framework:
• Start with an initial data structure D0.
• Operation i transforms Di–1 to Di.
• The cost of operation i is ci.
• Define a potential function  : {Di}  R,
such that (D0 ) = 0 and (Di )  0 for all i.
• The amortized cost ĉi with respect to  is
defined to be ĉi = ci + (Di) – (Di–1).

47. Understanding potentials
ĉi = ci + (Di) – (Di–1)
potential difference i
• If i > 0, then ĉi > ci. Operation i stores
work in the data structure for later use.
• If i < 0, then ĉi < ci. The data structure
delivers up stored work to help pay for
operation i.

48. The amortized costs bound the
true costs
The total amortized cost of n operations is
n n
ĉi  ci  (Di )  (Di1)
i1 i1
Summing both sides.

49. The amortized costs bound the
true costs

50. The amortized costs bound the
true costs

61. Conclusions
• Amortized costs can provide a clean abstraction
of data-structure performance.
• Any of the analysis methods can be used when
an amortized analysis is called for, but each
method has some situations where it is arguably
the simplest.
• Different schemes may work for assigning
amortized costs in the accounting method, or
potentials in the potential method, sometimes
yielding radically different bounds.