01 intro to algorithm--updated 2015

Instructor: Sadia Arshid
ANALYSIS OF ALGORITHM
Introduction to Algorithms

 Detailed study of the basic notions of the design of
algorithms and the underlying data structures
 Several measures of complexity are introduced
 Emphasis on the structure, complexity, and efficiency
of algorithms.
 Prepare the Students for Analyzing algorithms solving
same problems & making them capable of writing
optimize algorithms

 Text Book:
The Design And Analysis of Algorithmby Anany Levitin
Fundamentals of Algorithmby Richard Neopolitan
 Reference Material:
AlgorithmDesign by Micheal T. Good Rich
Introduction to Algorithms /2E, T. H. Cormen, C. E. Leiserson, and
R. L. Rivest, MIT Press, McGraw-Hill, New York, NY, 2001.
 Grading policy

Assignments 5%
Quizzes 10%
Project 15%
Midterm 20%
Final 50%

WHAT IS AN ALGORITHM?
 An algorithm is a set of rules for carrying out calculation
either by hand or on a machine.
 An algorithm is a finite step-by-step procedure to
achieve a required result.
 An algorithm is a sequence of computational steps that
transform the input into the output.
 An algorithm is a sequence of operations performed on
data that have to be organized in data structures.
 An algorithm is an abstraction of a program to be
executed on a physical machine (model of

An algorithmis a sequence of unambiguous
instructions for solving a problem, i.e., for obtaining
a required output for any legitimate input in a finite
amount of time.

Problem
algorithm
“computer”input output

ALGORITHM
• Problem
• problem instance and solution
• Sort list of n numbers in ascending order
• Determine whether x is in list of n numbers
• Algorithm is a general solution which solves all
instances of a problem.

• Generality
• Finiteness
• Non-ambiguity
• Efficiency
What properties an algorithm should have ?

• The algorithm applies to all instances of input data not only
for particular instances
Example:
Let’s consider the problem of increasingly sorting a
sequence of values.
For instance:
(2,1,4,3,5) (1,2,3,4,5)
input data result
Generality

Method:
Generality (cont’d)
2 1 4 3 5Step 1:
1 2 4 3 5
1 2 4 3 5
1 2 3 4 5
Step 2:
Step 3:
Step 4:
Description:
- compare the first two
elements:
if there are not in the desired
order swap them
- compare the second and the
third element and do the same
…..
- continue until the last two
elements were compared
The sequence has been ordered

GENERALITY (CONT’D)
• Is this algorithm a general one ? I meant, does it
assure the ordering of ANY sequence of values ?
• Answer: NO
Counterexample:
3 2 1 4 5
2 3 1 4 5
2 1 3 4 5
2 1 3 4 5
In this case the method doesn’t work, it isn’t a general sorting
algorithm

FINITENESS
• An algorithm have to terminate, i.e. to stop after a finite
number of steps
Example
Step1: Assign 1 to x;
Step2: Increase x by 2;
Step3: If x=10 then STOP;
else GO TO Step 2
How does this algorithm work ?

FINITENESS (CONT’D)
How does this algorithm work and what does it produce?
Step3: If x=10
then STOP;
else Print x; GO TO Step 2;
x=1
x=3 x=5 x=7 x=9 x=11
The algorithm generates odd numbers but it never stops !

FINITENESS (CONT’D)
The algorithm which generate all odd naturals smaller
than 10:
Step3: If x>=10
then STOP;
else Print x; GO TO Step 2

NON-AMBIGUITY
The operations in an algorithm must be rigorously specified:
• At the execution of each step one have to know exactly which is the
next step which will be executed
Example:
Step 1: Set x to value 0
Step 2: Either increment x with 1 or decrement x with 1
Step 3: If x∈[-2,2] then go to Step 2; else Stop.
As long as does not exist a criterion by which one can decide if
x is incremented or decremented, the sequence above cannot be

NON-AMBIGUITY (CONT’D)
Let’s modify the previous algorithm as follows:
Step 1: Set x to value 0
Step 2: Flip a coin
Step 3: If one obtains head
then increment x with 1
else decrement x with 1
Step 3: If x∈[-2,2] then go to Step 2, else Stop.
• This time the algorithm can be executed but … different executions can be
different

EFFICIENCY
• The need to design efficient algorithms
• What is efficiency?
• Example
• Sequential search Vs. Binary Search

ALGORITHM ANALYSIS
• How to analyze an algorithm?
• Predicting the resources that the algorithm requires.
• Memory
• Communications Bandwidth
• Logic gates etc
• Most important is Computational Time

ALGORITHM ANALYSIS
• Important thing before analysis
• Model of the machine upon which the algorithms is
executed.
• Random Access Machine (RAM) (Sequential)
• Running Time: No. of primitive operations or
“steps executed”.Instructor: Sadia Arshid

RUNNING TIME
 Most algorithms transform
input objects into output
objects.
 The running time of an
algorithm typically grows
with the input size.
 Average case time is often
difficult to determine.
 We focus on the worst
case running time.
Easier to analyze
Crucial to applications such
as games, finance and
0
20
40
60
80
100
120
RunningTime
1000 2000 3000 4000
I nput Size
best case
average case
worst case

EXPERIMENTAL STUDIES
 Write a program
implementing the
algorithm
 Run the program with
inputs of varying size and
composition
 Use a method in “Time.h”
to get an accurate
measure of the actual
running time
 Plot the results
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 50 100
I nput Size
Time(ms)

LIMITATIONS OF EXPERIMENTS
• It is necessary to implement the algorithm, which may
be difficult
• Results may not be indicative of the running time on
other inputs not included in the experiment.
• In order to compare two algorithms, the same hardware
and software environments must be used

THEORETICAL ANALYSIS
• Uses a high-level description of the algorithm instead
of an implementation
• Characterizes running time as a function of the input
size, n.
• Takes into account all possible inputs
• Allows us to evaluate the speed of an algorithm
independent of the hardware/software environment

ALGORITHM ANALYSIS
• Running Time
• We need a measure that is independent of computer,
programming language and the programmer.
• Concept of basic operation
• In sorting comparison is a basic operation.

ALGORITHM ANALYSIS
• How much time each construct / keyword of a
pseudo code takes to execute. Assume it takes ti
(the ith construct)
• Sum / Add up the execution time of all the
constructs / keywords. if there are m constructs
then
∑=
=
m
i
itT
1

ALGORITHM ANALYSIS
• That will be the execution time for a given input size
(say n)
∑=
=
m
i
in tT
1
♦Running time as the function of the input size
T(n)

ALGORITHM ANALYSIS
• What are the constructs / Keywords.
• Time for each construct
• Total Time
• Total time as a function of input size

ALGORITHM ANALYSIS
• Construct:
• Sequence
• Selection
• Iterations
• Recursion

ALGORITHM ANALYSIS
 Sequence Statements: Just add the running time of the
statements
 If-Then-Else: if (condition) S1 else S2
Running time of the test plus the larger of the running times
of S1 and S2.
 Iteration is at most the running time of the statements
inside the loop, including the times the number of
iterations.
 Nested Loops: Analyze these inside out. The total Running
time of a statement inside a group of nested loops is the
running time of the statement multiplied by the product of
the size of all the loops.
 Function Calls: Analyzing from inside to out. If there
are function calls, these must be analyzed first.

EXAMPLE
Algorithm arrayMax(A, n)
Input array A of n integers
Output maximum element of A
currentMax ← A[0]
for i ← 1 to n − 1 do
if A[i] > currentMax then
currentMax ← A[i]
return currentMax
Example: find max element of
an array

COUNTING PRIMITIVE
OPERATIONS
 By inspecting the pseudocode, we can determine the
maximum number of primitive operations executed by an
algorithm, as a function of the input size
AlgorithmarrayMax(A, n) # operations
currentMax ← A[0] 2
fori ← 1 to n − 1 do 1 + n
if A[i] > currentMax then 2(n − 1)
currentMax ← A[i] 2(n − 1)
{ increment counter i } 2(n − 1)
return currentMax 1
Total 7n − 2

ESTIMATING RUNNING TIME
• Algorithm arrayMax executes 7n− 2 primitive operations in the worst
case.
• This notation is complex to use. To simplify it we will be using order of
growth that is dominating term of expression; which is n (will be
discuss in detail later)
• Instead of computing running time of every statement of the algorithm
we can further simplify it by calculating running time of basic
operation only(Time Complexity) (will be discuss in detail later)
• In This example
• basic operation is comparison
• Executed n-1 number of times
• Order of growth is n
• Same as order of growth of computing running time of every statement.Instructor: Sadia Arshid

IMPORTANT PROBLEM TYPES
• Sorting
• Sorting keys
• Stable
• Inplace
• Searching
• String processing

IMPORTANT PROBLEM TYPES
• Graph problems
• Combinatorial Problems
• Geometric Problems
• Numerical Problems

FUNDAMENTAL DATA
STRUCTURES Linear data Structures
Arrays
Link list
Stack
Queue
 Graphs
Adjacency Matrix
Adjacency Link list
 Trees
 Set & Dictionaries

01 intro to algorithm--updated 2015

More Related Content

What's hot

Similar to 01 intro to algorithm--updated 2015

More from Hira Gul

Recently uploaded

01 intro to algorithm--updated 2015