This document provides an overview of algorithms and their analysis. It defines an algorithm as a finite sequence of unambiguous instructions that will terminate in a finite amount of time. Key aspects that algorithms must have are being input-defined, having output, being definite, finite, and effective. The document then discusses steps for designing algorithms like understanding the problem, selecting data structures, and verifying correctness. It also covers analyzing algorithms through evaluating their time complexity, which can be worst-case, best-case, or average-case, and space complexity. Common asymptotic notations like Big-O, Omega, and Theta notation are explained for describing an algorithm's efficiency. Finally, basic complexity classes and their properties are summarized.

1. Analysis & Design of
Algorithm
Unit - 1

2. What is an Algorithm?

3. Definition
• An Algorithm is a finite sequence of instructions, each of which has a
clear meaning and can be performed with a finite amount of effort in
a finite length of time.
• No matter what the input values may be, an algorithm terminates
after executing a finite number of instructions.

4. Criteria for Algorithm
• Input:
• There are zero or more quantities, which are externally supplied;
• Output:
• At least one quantity is produced;
• Definiteness:
• Each instruction must be clear and unambiguous;

5. • Finiteness:
• If we trace out the instructions of an algorithm, then for all cases the
algorithm will terminate after a finite number of steps;
• Effectiveness:
• Every instruction must be sufficiently basic that it can in principle be
carried out by a person using only pencil and paper. It is not enough
that each operation be definite, but it must also be feasible.

6. Characteristics of an Algorithm

7. DESIGNING ALGORITHMS

8. Steps
1. Understanding the Problem
2. Ascertain the capabilities of the computational device
3. Selection of Exact or Approximate Solutions
4. Decide on the appropriate data structure
5. Algorithm Design Techniques

9. 6. Methods of Specifying an Algorithm
7. Verifying an Algorithm / Proving Correctness of an Algorithm
8. Analyzing an Algorithm
9. Implementation or Coding of an Algorithm

10. 1. Understanding the problem:
• The problem given should be understood completely.
• Check if it is similar to some standard problems & if a Known algorithm
exists.
• Otherwise a new algorithm has to be devised.
• Creating an algorithm is an art which may never be fully automated.
• An important step in the design is to specify an instance of the problem.

11. 2. Ascertain the capabilities of the
computational device:
• Once a problem is understood we need to Know the capabilities of
the computing device this can be done by Knowing the type of the
architecture, speed & memory availability

12. 3. Selection of Exact or Approximate Solutions
• Once algorithm is devised, it is necessary to show that it computes answer
for all the possible legal inputs.
• The solution is stated in two forms, Exact solution or Approximate solution.
• Examples of problems where an exact solution cannot be obtained are
i) Finding a square root of number.
ii) Solutions of non linear equations.

13. 4. Decide on the appropriate data structure:
• Some algorithms do not demand any ingenuity in representing their inputs.
• Some others are in fact are predicted on ingenious data structures.
• A data type is a well-defined collection of data with a well-defined set of
operations on it.
• A data structure is an actual implementation of a particular abstract data
type.

14. 5. Algorithm design techniques
• Creating an algorithm is an art which may never be fully automated.
• By mastering these design strategies, it will become easier for you to
devise new and useful algorithms.
• Divide & Conquer, Dynamic programming , Greedy strategy , such
technique.

15. 6. Methods of specifying an algorithm
• There are mainly two options for specifying an algorithm:
• Use of Natural Language or Pseudocode, &
• Flowcharts.
• A Pseudo code is a mixture of natural language & programming language
like constructs.
• A flowchart is a method of expressing an algorithm by a collection of
connected geometric shapes.

16. 7. Proving an algorithms correctness
• Once algorithm is devised, it is necessary to show that it computes
answer for all the possible legal inputs.
• We refer to this process as algorithm validation.
• The process of validation is to assure us that this algorithm will work
correctly independent of issues concerning programming language it
will be written in.

17. 8. Analyzing algorithms
• As an algorithm is executed, it uses the computers central processing
unit to perform operation and its memory (both immediate and
auxiliary) to hold the program and data.
• Analysis of algorithms and performance analysis refers to the task of
determining how much computing time and storage an algorithm
requires.

18. Analysis of Algorithms

19. What is it?
• There are many criteria upon which we can judge an algorithm for
instance:
1. Does it do what we want to do?
2. Does it work correctly according to the original specifications to the task?
3. Is there documentation that describes how to use it and how it works?
4. Are procedures created in such a way that they perform logical sub functions?
5. Is the code readable?

20. ALGORITHM COMPLEXITY
• The complexity of an algorithm f(n) gives the running time and / or
storage space required by the algorithm in terms of n as the size of
input data.
• Suppose X is an algorithm and n is the size of input data, the time and
space used by the Algorithm X are the two main factors which decide
the efficiency of X.

21. • Time Factor:
• The time is measured by counting the number of key operations such
as comparisons in sorting algorithm
• Space Factor:
• The space is measured by counting the maximum memory space
required by the algorithm

22. SPACE COMPLEXITY
• Space complexity of an algorithm represents the amount of memory
space required by the algorithm in its life cycle.
• Space required by an algorithm is equal to the sum of the following
two components −

23. • A fixed part that is a space required to store certain data and variables, that are
independent of the size of the problem. For example simple variables & constant
used, program size etc.
• A variable part is a space required by variables, whose size depends on the size of
the problem. For example dynamic memory allocation, recursion stack space etc.

24. • Space complexity S(P) of any algorithm P is: -
• S(P) = C + SP(I)
• Where C is the fixed part and
• S(I) is the variable part of the algorithm which depends on instance
characteristic I.

25. Example: -
• Algorithm: SUM(A, B)
• Step 1 - START
• Step 2 - C ← A + B +10
• Step 3 - Stop

26. Explanation: -
• Here we have three variables A, B and C and one constant.
• Hence S(P) = 1+3 =4.
• Now space depends on data types of given variables and constant
types and it will be multiplied accordingly.

27. Components of Space
• Instruction space: Instruction space is the space needed to store the
compiled version of the program instructions.
• Data space: Data space is the space needed to store all constant and
variable values. Data space has two components:
• Space needed by constants and simple variables in program.
• Space needed by dynamically allocated objects such as arrays and class
instances.

28. • Environment stack space: The environment stack is used to save
information needed to resume execution of partially completed functions.
• Instruction Space: The amount of instructions space that is needed
depends on factors such as:
• The compiler used to complete the program into machine code.
• The compiler options in effect at the time of compilation
• The target computer.

29. TIME COMPLEXITY
• Time Complexity of an algorithm represents the amount of time
required by the algorithm to run to completion.
• Time requirements can be defined as a numerical function T(n).
• Where T(n) can be measured as the number of steps, provided each
step consumes constant time.

30. Example:
• Addition of two n-bit integers takes n steps. Consequently, the total
computational time is T(n) = c*n,
• where c is the time taken for addition of two bits.
• Here, we observe that T(n) grows linearly as input size increases.
• The time T (P) taken by a program P is sum of compile time and run
time.

31. • The compile time does not depend on the instance characteristics.
• Also, we may assume that a compiled program will be run several time of a
program. This run time is denoted by tp.
• Because of many of the factor tp depends on are not known at the time of
a program is conceived, it is reasonable to attempt only to estimate tp .
• T(p)=c+tp

32. WORST-CASE, BEST-CASE, AVERAGE CASE
EFFICIENCIES
• Algorithm efficiency depends on the input size n.
• And for some algorithms efficiency depends on type of input.
• We have best, worst & average case efficiencies.

33. Worst-case efficiency:
• Efficiency (number of times the basic operation will be executed) for
the worst case input of size n. i.e. The algorithm runs the longest
among all possible inputs of size n.

34. Best-case efficiency:
• Efficiency (number of times the basic operation will be executed) for
the best case input of size n. i.e. The algorithm runs the fastest among
all possible inputs of size n.

35. Average-case efficiency:
• Average time taken (number of times the basic operation will be
executed) to solve all the possible instances (random) of the input.
• NOTE: NOT the average of worst and best case

36. Basic Efficiency classes
• The time efficiencies of a large number of algorithms fall into only a
few classes.
Constant O(1)
Logarithmic O(log n)
Linear O(n)
n Log n O(n log n)
Quadratic O(n2)
Cubic O(n3)
Polynomial nO(1)
Exponential 2O(n)

37. Efficiency Class
1 Constant
Log (n) Logarithmic
n Linear
n Log (n) n Log (n)
n2 Quadratic
n3 Cubic
2n Exponential
n! Factorial
FAST
SLOW
HIGH TIME EFFICIENCY
LOW TIME EFFICIENCY

38. ALGORITHM ANALYSIS
• Efficiency of an algorithm can be analyzed at two different stages, before
implementation and after Implementation, as follows−
• A priori analysis − This is theoretical analysis of an algorithm. Efficiency of
algorithm is measured by assuming that all other factors e.g. processor
speed, are constant and have no effect on implementation.

39. • A posterior analysis − This is empirical analysis of an algorithm. The
selected algorithm is implemented using programming language.
• This is then executed on target computer machine.
• In this analysis, actual statistics like running time and space required,
are collected.

40. • We learn a priori algorithm analysis. Algorithm analysis deals with
the execution or running time of various operations involved.
• Running time of an operation can be defined as no. of computer
instructions executed per operation.

41. Asymptotic Notations
• Asymptotic analysis of an algorithm, refers to defining the
mathematical boundation / framing of its run-time performance.
• Using asymptotic analysis, we can very well conclude the best case,
average case and worst case scenario of an algorithm.

42. • Asymptotic analysis are input bound i.e., if there's no input to the
algorithm it is concluded to work in a constant time. Other than the "input"
all other factors are considered constant.
• Asymptotic analysis refers to computing the running time of any operation
in mathematical units of computation.
• For example, running time of one operation is computed as f(n) and may
be for another operation it is computed as g(n2).

43. • Which means first operation running time will increase linearly with
the increase in n and running time of second operation will increase
exponentially when n increases.
• Similarly the running time of both operations will be nearly same if n
is significantly small.

44. • Usually, time required by an algorithm falls under three types: -
• Best Case: minimum time required for program execution.
• Average Case: average time required for program execution.
• Worst Case: maximum time required for program execution.

45. • Following are commonly used asymptotic notations used in
calculating running time complexity of an algorithm.
• Ο Notation (Big Oh)
• Ω Notation (Omega)
• θ Notation (Theta)

46. Big Oh Notation, Ο
• The Ο(n) is the formal way to express the upper bound of an
algorithm's running time.
• It measures the worst case time complexity or longest amount of time
an algorithm can possibly take to complete.

47. Definition:
• A function t(n) is said to be in O(g(n)), denoted t(n) ∈O(g(n)), if t(n) is
bounded above by some constant multiple of g(n) for all large n, i.e.,
if there exist some positive constant c and some non-negative integer
n0 such that,
• t(n) ≤ cg(n) for all n ≥ n0

49. • For example,
• for a function f(n)
• Ο(f(n)) = { g(n) : there exists c > 0 and n0 such that g(n) ≤ c.f(n) for all
n > n0. }

50. Omega Notation, Ω
• The Ω(n) is the formal way to express the lower bound of an
algorithm’s running time.
• It measures the best case time complexity or best amount of time an
algorithm can possibly take to complete.

51. Definition:
• A function t(n) is said to be in Ω (g(n)), denoted t(n) ϵ Ω (g(n)), if t(n) is
bounded below by some constant multiple of g (n) for all large n, i.e.,
if there exist some positive constant c and some nonnegative integer
n0 such that,
• t(n) ≥ cg(n) for all n ≥ n0

53. • For example, for a function f(n)
• Ω(f(n)) ≥ { g(n) : there exists c > 0 and n0 such that g(n) ≤ c.f(n) for all
n > n0. }

54. Theta Notation, θ
• The θ(n) is the formal way to express both the lower bound and
upper bound of an algorithm's running time.

55. Definition:
• A function t (n) is said to in Θ (g(n)), denoted t(n) ∈ Θ (g(n), if t(n) is
bounded both above and below by some constant multiple of g (n)
for all large n, i.e., if there exist some positive constant c1 and c2 and
some non-negative integer n0 such that
• c2 g(n) ≤ t(n) ≤ c1 g(n) for all n ≥ n0

57. • Θ(f(n)) = { g(n) if and only if g(n)= Ο(f(n)) and g(n) = Ω(f(n)) for all n >
n0. }

58. Basic Efficiency classes

59. FAST Efficiency Class HIGH TIME EFFICIENCY
1 Constant
Log N Logarithmic
N Linear
N Log N N Log N
N2 Quadratic
N3 Cubic
2N Exponential
SLOW N! Factorial LOW TIME EFFICIENCY

60. Constant O (1)
Logarithmic O (Log N)
Linear O (N)
N Log N O (N Log N)
Quadratic O (N2)
Cubic O (N3)
Polynomial N O(1)
Exponential 2 O(n)

61. Analyzing Algorithms

62. • Suppose ‘M’ is an algorithm, and suppose ‘n’ is the size of the input
data. Clearly the complexity of f(n) of M increases as n increases. It is
usually the rate of increase of f(n) we want to examine.
• This is usually done by comparing f(n) with some standard functions.
• The most common computing times are:

63. • O(1), O(log n), O(n), O(n log n), O(n2), O(n3), O(2n), n! and nn

64. Numerical Comparison of Different
Algorithms
n log n n log n n2 n3 nn
1 0 0 1 1 2
2 1 2 4 8 4
4 2 8 16 64 16
8 3 24 64 512 256
16 4 64 256 4096 65,536
32 5 160 1024 32,768 4,294,967,296
64 6 384 4096 2,62,144 Note 1
128 7 896 16,384 2,097,152 Note 2
256 8 2048 65,536 1,677,216 ????????