The document discusses algorithms and their analysis. It defines an algorithm as a sequence of unambiguous steps to solve a problem within a finite time. Characteristics of algorithms include being unambiguous, having inputs/outputs, and terminating in finite time. Algorithm analysis involves determining theoretical and empirical time and space complexity as input size increases. Time complexity is analyzed by counting basic operations, while space complexity considers fixed and variable memory usage. Worst, best, and average cases analyze how efficiency varies with different inputs. Asymptotic analysis focuses on long-term growth rates to compare algorithms.

1. Fundamentals of Algorithms and
Data Structures
Module 1

2. Algorithm Development
• Development of Algorithm & Flowcharts is first step towards learning programming language.
Generally you need to develop algorithm/flowchart of some process with step by step
procedure.
• What is Algorithm?
• An algorithm is a finite sequence of well-defined steps or operations for solving a problem.
• An algorithm generally takes some input, carries out a number of effective steps in a finite
amount of time, and produces some output.
08-09-2022 2

3. Preamble:
08-09-2022 3

4. Algorithm
• An Algorithm is a sequence of unambiguous instructions for
solving a problem,
• i.e., for obtaining a required output for any legitimate input in a finite
amount of time.

5. • Algorithm is a step-by-step procedure, which defines a set of instructions to
be executed in a certain order to get the desired output. Algorithms are
generally created independent of underlying languages, i.e. an algorithm
can be implemented in more than one programming language.
• From the data structure point of view, following are some important
categories of algorithms −
• Search − Algorithm to search an item in a data structure.
• Sort − Algorithm to sort items in a certain order.
• Insert − Algorithm to insert item in a data structure.
• Update − Algorithm to update an existing item in a data structure.
• Delete − Algorithm to delete an existing item from a data structure.

6. Characteristics of an Algorithm
• Not all procedures can be called an algorithm. An algorithm should have the
following characteristics −
• Unambiguous − Algorithm should be clear and unambiguous. Each of its steps (or
phases), and their inputs/outputs should be clear and must lead to only one
meaning.
• Input − An algorithm should have 0 or more well-defined inputs.
• Output − An algorithm should have 1 or more well-defined outputs, and should
match the desired output.
• Finiteness − Algorithms must terminate after a finite number of steps.
• Feasibility − Should be feasible with the available resources.
• Independent − An algorithm should have step-by-step directions, which should be
independent of any programming code.

7. Notion of algorithm
“computer”
problem
algorithm
input output

8. PSEUDOCODE
 Pseudocode (pronounced SOO-doh-kohd) : readable description of what a computer program or
algorithm must do, expressed in a formally-styled natural language rather than in a
programming language.
 Detailed step in the process of developing a program.
 Express the design in great detail and provides programmers a detailed template for the next step
of writing code in a specific programming language.
PSEUDOCODE
FOR i <-- 1 TO 100 DO
IF i is divisible by 3 AND i is divisible by 5 THEN
OUTPUT “Hiiiii"
ELSE IF i is divisible by 3 THEN
OUTPUT “Haiii"
ELSE IF i is divisible by 5 THEN
OUTPUT “Hello"
ELSE
OUTPUT i
CODE
for i in range(1,100):
if i % 3 == 0 and i % 5 == 0:
print(‘Hiiiii')
elif i % 3 == 0:
print(‘Haiii')
elif i % 5 == 0:
print(‘Hello')
else:
print(i)

9. Formatting and Conventions in Pseudocoding
 INDENTATION in pseudocode should be identical to its implementation in a
programming language. Try to indent at least four spaces.
 The pseudocode entries are to be cryptic, AND SHOULD NOT BE PROSE. NO
SENTENCES.
 No flower boxes in pseudocode.
 Do not include data declarations in pseudocode.

10. Fundamentals ofAnalysis of
algorithm efficiency

11. Characteristics of a Data Structure
• Correctness − Data structure implementation should implement its
interface correctly.
• Time Complexity − Running time or the execution time of operations
of data structure must be as small as possible.
• Space Complexity − Memory usage of a data structure operation
should be as little as possible.

12. Analysis of algorithms
 Issues:
 Correctness - Does it work as advertised?
 time efficiency - Are time requirements minimized?
 space efficiency - Are space requirements minimized?
 Optimality - Do we have the best balance between minimizing
time and space?
 Approaches:
 Theoretical analysis
 Empirical analysis

13. Theoretical analysis of time
efficiency
Time efficiency is analyzed by determining the
number of repetitions of the basic operation as a
function of input size
 Basic operation: the operation that contributes
the most towards the running time of the
algorithm

14. Input size and basic operation examples
Problem Input size measure Basic operation
Searching for key
in a list of n items
Number of list’s
items, i.e. n
Key comparison
Multiplication of
two matrices
Matrix dimensions or
total number of
elements
Multiplication of
two numbers
Checking primality
of a given integer n
n’size = number of
digits (in binary
representation)
Division
Typical graph
problem
#vertices and/or
edges
Visiting a vertex
or traversing an
edge

15. Empirical analysis of time efficiency
⦿Select a specific (typical) sample of inputs
⦿Use physical unit of time (e.g., milliseconds)
or
Count actual number of basic operation’s
executions
⦿Analyze the empirical data

16. Algorithm Complexity
• Suppose X is treated as an algorithm and N is treated as the size of input data, the time and
space implemented by the Algorithm X are the two main factors which determine the
efficiency of X.
• Time Factor − The time is calculated or measured by counting the number of key
operations such as comparisons in sorting algorithm.
• Space Factor − The space is calculated or measured by counting the maximum memory
space required by the algorithm. Space complexity S(p) of any algorithm p is S(p) = A +
Sp(I) Where A is treated as the fixed part and S(I) is treated as the variable part of the
algorithm which depends on instance characteristic I. Following is a simple example that
tries to explain the concept
• The complexity of an algorithm f(N) provides the running time and / or storage space
needed by the algorithm with respect of N as the size of input data.
SUM(P, Q)
Step 1 - START
Step 2 - R ← P + Q + 10
Step 3 - Stop

17. Space Complexity
• Space complexity of an algorithm represents the amount of memory
space required by the algorithm in its life cycle. The space required by an
algorithm is equal to the sum of the following two components −
• A fixed part that is a space required to store certain data and variables,
that are independent of the size of the problem. For example, simple
variables and constants used, program size, etc.
• A variable part is a space required by variables, whose size depends on
the size of the problem. For example, dynamic memory allocation,
recursion stack space, etc.
• Space complexity S(P) of any algorithm P is S(P) = C + SP(I), where C is
the fixed part and S(I) is the variable part of the algorithm, which
depends on instance characteristic I

18. Time Complexity
• Time complexity of an algorithm represents the amount of time
required by the algorithm to run to completion.
• Time requirements can be defined as a numerical function T(n),
where T(n) can be measured as the number of steps, provided each
step consumes constant time.
• For example, addition of two n-bit integers takes n steps.
Consequently, the total computational time is T(n) = c ∗ n, where c is
the time taken for the addition of two bits. Here, we observe that T(n)
grows linearly as the input size increases.

19. Efficiency- Execution Time Cases
• There are three cases which are usually used to compare various
data structure's execution time in a relative manner.
• Worst Case − This is the scenario where a particular data structure
operation takes maximum time it can take. If an operation's worst
case time is ƒ(n) then this operation will not take more than ƒ(n) time
.
• Average Case − This is the scenario depicting the average execution
time of an operation of a data structure. If an operation takes ƒ(n)
time in execution, then m operations will take mƒ(n) time.
• Best Case − This is the scenario depicting the least possible execution
time of an operation of a data structure.
For example, the best case for a simple linear search on a list
occurs when the desired element is the first element of the list.

20. Efficiencies
 Worst Case Efficiency:
 Is its efficiency for the worst case input of size n, which is an
input of size n for which the algorithm runs the longest among
all possible inputs of that size.
 Cworst(n)
 Best-case efficiency:
 Is its efficiency for the Best case input of size n, which is an
input of size n for which the algorithm runs the fastest among
all possible inputs of that size.
 Cbest(n)

21. Best-case, average-case, worst-case
For some algorithms, efficiency depends on form of input:
 Worst case: Cworst(n) – maximum over inputs of size n
 Best case: Cbest(n) – minimum over inputs of size n
 Average case: Cavg(n) – “average” over inputs of size n
 Number of times the basic operation will be executed on typical input.
 NOT the average of worst and best case.
 Expected number of basic operations considered as a random variable under some
assumption about the probability distribution of all possible inputs. So, avg = expected
under uniform distribution.

22. Example: Sequential search
 Worst case
 Best case
 Average case
n key comparisons
1 comparisons
(n+1)/2, assuming K is inA

23. Order of growth
• Most important: Order of growth within a
constant multiple as n→∞
• Example:
– How much faster will algorithm run on computer that
is twice as fast?
– How much longer does it take to solve problem of
double input size?

24. Values of some important functions as n  

25. Asymptotic Analysis
• Implement both the algorithms (1 AND 2) and run the two programs on your computer for different
inputs and see which one takes less time. There are many problems with this approach for analysis
of algorithms.
1) It might be possible that for some inputs, first algorithm performs better than the second. And for
some inputs second performs better.
2) It might also be possible that for some inputs, first algorithm perform better on one machine and the
second works better on other machine for some other inputs.
Asymptotic Analysis : evaluate the performance of an algorithm in terms of input size (we don’t
measure the actual running time). We calculate, how the time (or space) taken by an algorithm
increases with the input size.
Example: Linear Search (order of growth is linear) and the other way is Binary Search (order of growth
is logarithmic)
• Our focus would be on finding the time complexity rather than space complexity, and by finding the
time complexity, we can decide which data structure is the best for an algorithm.
• Suppose we have an array of 100 elements, and we want to insert a new element at the beginning
of the array. This becomes a very tedious task as we first need to shift the elements towards the
right, and we will add new element at the starting of the array.

26. • The running time of an algorithm depends on how long it takes a
computer to run the lines of code of the algorithm—and that
depends on the speed of the computer, the programming
language, and the compiler that translates the program from the
programming language into code that runs directly on the
computer, among other factors.
• Efficiency of the algorithm with the running depends on the size of
the input .
• That is, we are concerned with how the running time of an
algorithm increases with the size of the input in the limit.

27. • Let's think about the running time of an algorithm more carefully.
• We can use a combination of two ideas.
• First, we need to determine how long the algorithm takes, in terms of
the size of its input. This idea makes intuitive sense.
• For example, the maximum number of guesses in linear search and
binary search increases as the length of the array increases.
• So we think about the running time of the algorithm as a function of the
size of its input.

28. • The second idea is that we must focus on how fast a function grows with the
input size.
• We call this the rate of growth of the running time.
• To keep things manageable, simplify the function to distill the most important
part and cast aside the less important parts.
• For example, suppose that an algorithm, running on an input of size n, takes
6n^2 + 100n + 300
The 6n^2, squared term becomes larger than the remaining terms, 100n + 300

29. We would say that the running time of this algorithm grows as n^2, dropping the
coefficient 6 and the remaining terms 100n + 300 really doesn’t matter what
coefficients we use; as long as the running time is an^2 + bn + c.
We understand that there will always be a value of n for which an^2 greater than
bn + c.
By dropping the less significant terms and the constant coefficients, we can
focus on the important part of an algorithm's running time—its rate of growth.
When we drop the constant coefficients and the less significant terms, we
use asymptotic notation.

30. Asymptotic Notation
• Asymptotic notations are of three forms
• Q - Big Theta, O – big O notation, W, - big Omega
• Defined for functions over the natural numbers.
• Ex: f(n) = Q(n2).
• Describes how f(n) grows in comparison to n2.
• Define a set of functions; in practice used to compare two function sizes.
• The notations describe different rate-of-growth relations between the
defining function and the defined set of functions.

31. Asymptotic Notations
• O (Big-Oh) - measures the performance of an algorithm by
simply providing the order of growth of the function.
• Ω (Big-Omega) –lower bound of an algorithm's running time
• Θ (Big-Theta) - average case scenarios

32. Asymptotic order of growth
A way of comparing functions that ignores constant factors
and small input sizes (because?)
 O(g(n)): class of functions f(n) that grow no faster than g(n)
 Θ(g(n)): class of functions f(n) that grow at same rate as
g(n)
 Ω(g(n)): class of functions f(n) that grow at least as fast as
g(n)

33. Q-notation- Big Theta
• Formal definition
– A function t(n) is said to be in Q(g(n)), denoted t(n)  Q(g(n)), if t(n) is
bounded both above and below by some positive constant multiples
of g(n) for all large n, i.e., if there exist some positive constant c1 and
c2 and some nonnegative integer n0 such that
c2 g(n)  t(n)  c1 g(n) for all n  n0

34. Q-notation
Q(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)
}
For function g(n), we define Q(g(n)),
big-Theta of n, as the set:
g(n) is an asymptotically tight bound for f(n).
Intuitively: Set of all functions that
have the same rate of growth as g(n).

35. Big-theta

36. Q-notation
Q(g(n)) = {f(n) :
 positive constants c1, c2, and n0,
such that n  n0,
we have 0  c1g(n)  f(n)  c2g(n)
}
For function g(n), we define Q(g(n)),
big-Theta of n, as the set:
f(n) and g(n) are nonnegative, for large n.

37. O-notation – Big O
Definition: A function t(n) is said to be in O(g(n)), denoted t(n)  O(g(n)) is
bounded above by some constant multiple of g(n) for all large n, i.e., there
exist positive constant c and non- negative integer n0 such that
f(n) ≤ c g(n) for every n ≥ n0
Example 1: f(n)=2n+3 , g(n)=n
First, we will replace f(n) by 2n+3 and g(n) by n.
2n+3 <= c.n
Let's assume c=5, n=1 then
2*1+3<=5*1 => 5<=5 (For n=1, the above
condition is true).
If n=2
2*2+3<=5*2 => 7<=10 (For n=2, the above
condition is true.)

38. O-notation
O(g(n)) = {f(n) :
 positive constants c and n0, such
that n  n0,
we have 0  f(n)  cg(n) }
For function g(n), we define O(g(n)),
big-O of n, as the set:
g(n) is an asymptotic upper bound for f(n).
Intuitively: Set of all functions whose rate of
growth is the same as or lower than that of
g(n).
f(n) = Q(g(n))  f(n) = O(g(n)).
Q(g(n))  O(g(n)).

39. W-notation- Big Omega
• Formal definition
– A function t(n) is said to be in W(g(n)), denoted t(n)  W(g(n)), if t(n) is
bounded below by some constant multiple of g(n) for all large n, i.e., if
there exist some positive constant c and some nonnegative integer n0
such that
t(n)  cg(n) for all n  n0

40. W -notation
g(n) is an asymptotic lower bound for f(n).
Intuitively: Set of all functions whose rate of
growth is the same as or higher than that of
g(n).
f(n) = Q(g(n))  f(n) = W(g(n)).
Q(g(n))  W(g(n)).
W(g(n)) = {f(n) :
 positive constants c and n0, such
that n  n0,
we have 0  cg(n)  f(n)}
For function g(n), we define W(g(n)),
big-Omega of n, as the set:

41. Big-omega

42. Establishing order of growth using limits
lim T(n)/g(n)
=
0 order of growth of T(n) < order of growth of g(n)
c > 0 order of growth of T(n) = order of growth of g(n)
∞ order of growth of T(n) > order of growth of g(n)
n→∞

43. Asymptotic Analysis
• In analysis of algorithms, asymptotic analysis is a method of
describing limiting behavior.
• Examples include the performance of algorithms when applied
to very large input data, or the behavior of physical systems
when they are very large.
• The simplest example is when we have a function f(n) and
wish to describe its properties when n becomes very large.
• Thus, if f(n) = n2+3n, the term 3n becomes insignificant
compared to n2 when n is very large.
We say that "f(n) is asymptotically equivalent to n2 as n
→ ∞" and write f(n) ~ n2.

44. Example
3n2 + 17
1<log n<n < n2 <n3 < ……… < 2n < 3n < ……< nn
W(1), W(n), W(n2)  lower bounds
• O(n2), O(n3), ...  upper bounds
• Q(n2)  exact bound

45. Relations Between Q, O, W

46. Relations Between Q, W, O
• I.e., Q(g(n)) = O(g(n))  W(g(n))
• In practice, asymptotically tight bounds are
obtained from asymptotic upper and lower bounds.
Theorem : For any two functions g(n) and f(n),
f(n) = Q(g(n)) iff
f(n) = O(g(n)) and f(n) = W(g(n)).

47. Best/Worst/Average Case
• In computer science, best, worst and average cases of a given algorithm express what the resource usage is at
least, at most and on average, respectively. Usually the resource being considered is running time, but it could
also be memory or other resources.
• In real-time computing, the worst-case execution time is often of particular concern since it is important to
know how much time might be needed in the worst case to guarantee that the algorithm will always finish on
time.
• Average performance and worst-case performance are the most used in algorithm analysis. Less widely found
is best-case performance, but it does have uses: for example, where the best cases of individual tasks are
known, they can be used to improve the accuracy of an overall worst-case analysis. Computer scientists use
probabilistic analysis techniques, especially expected value, to determine expected running times.

48. Best/Worst/Average Case
• Linear search on a list of n elements. In the worst case, the search
must visit every element once. This happens when the value being
searched for is either the last element in the list, or is not in the list.
However, on average, assuming the value searched for is in the list
and each list element is equally likely to be the value searched for, the
search visits only n/2 elements.

49. Some properties of asymptotic order of growth
 f(n)  O(f(n))
 f(n)  O(g(n)) iff g(n) W(f(n))
 If f (n)  O(g (n)) and g(n)  O(h(n)) , then f(n) 
O(h(n))
Note similarity with a ≤ b
 If f1(n)  O(g1(n)) and f2(n)  O(g2(n)) , then
f1(n) + f2(n)  O(max{g1(n), g2(n)})
Also, 1in Q(f(i)) = Q (1in f(i))

50. Basic asymptotic efficiency classes
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial

51. Plan for analyzing nonrecursive algorithms
 Decide on parameter n indicating input size
 Identify algorithm’s basic operation
 Determine worst, average, and best cases for input of
size n
 Set up a sum for the number of times the basic
operation is executed
 Simplify the sum using standard formulas and rules
(see Appendix A)

52. Useful summation formulas and rules
lin1 = 1+1+…+1 = n - 1 + 1
In particular, 1in1 = n - 1 + 1 = n  Q(n)
1in i = 1+2+…+n = n(n+1)/2  n2/2  Q(n2)
1in i2 = 12+22+…+n2 = n(n+1)(2n+1)/6  n3/3  Q(n3)
0in ai = 1 + a +…+ an = (an+1 - 1)/(a - 1) for any a  1
In particular, 0in 2i = 20 + 21 +…+ 2n = 2n+1 - 1 
Q(2n )
(ai ± bi ) = ai ± bi cai = cai
+ m+1iuai
liuai = limai

53. Example 1: Maximum element
T(n) = 1in-1 1 = n-1 = Q(n) comparisons

54. Example 2: Element uniqueness
problem
T(n) = 0in-2 (i+1jn-1 1)
= 0in-2 n-i-1 = (n-1+1)(n-1)/2
= Q( n2) comparisons

55. Example 3: Matrix multiplication
T(n) = 0in-1 0in-1 n
= 0in-1 Q( n2
)
= Q( n3 ) multiplications