Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com 1
Data Structure Advanced

Algorithm
2
Well-defined sequence of computational steps.
The word Algorithm refers to a precise method
useable by a computer for solution of a problem.
The Algorithm accepts a set of values as input
and produces a set of values as output.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Examples
3
Algorithm written in Human language
Input a,b,c
Compute the sum of a,b,c
Divide the Sum by 3
Display the Result
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Example
4
Algorithm written in Programming language.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Example
5
Algorithm written in Pseudocode Syntax.
Step 1: Promt a,b,c
Step 2: Let Sum a+b+c
Step 3: Let Avg Sum / 3
Step 4: Output Avg
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

ALGORITHM
6
What do we mean by input ?
The word input interdependent on the type of
problem.
- Set of values or elements
e.g in the array problem
- A Set of bits
e.g in the string problem
- A Set of objects
e.g in the graph problems the input may be the set
of vertices and edges.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

The Properties of Algorithm
7
It must be correct
It must be composed of a series of concrete
steps.
There can be no ambiguity as to which step will
be performed next.
It must be terminate.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Analysis of Algorithm
8
What is a good algorithm ?
- Is a clear algorithm ??
- Has a minimum steps ??
- Decreases the CPU time
- Has a minimum memory (RAM) space ?
- It is a good mapping for the problem.
- Uses minimum spaces of disk I/O
** In this course the best Algorithm is the one that
uses the minimum CPU resources
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Study of Algorithm
9
Design Analysis
Methods and technique which Mathematical comparison of
yield a good and useful algorithm ( without
Algorithm to solve a problem. actually implement )
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Algorithms
10
Types , methods , and techniques of ALG :
- Incremental Algorithms
- Simple Recursive Algorithms
- Divide and Conquer Algorithms
- Greedy Algorithms
- Dynamic Algorithms
- Backtracking Algorithms
- Branch and Bound Algorithms
- Brute and Force Algorithms
- Randomized Algorithms
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Analysis of Algorithm
11
The analysis of an algorithm focuses on the time
and space complexity.
The Time Complexity is the amount of time
required by an algorithm to run to completion.
The Time Complexity is a function of input size
“n” T(n)
The Time Complexity refers to the amount of time
required by an ALG to run completion
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
12
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
13
Different times can arise for the same algorithm
- Worse-Case time complexity
is a function defined by the maximum amount of time needed by an
algorithm for an input size “n”.
- Best-Case time complexity
is a function defined by the minimum amount of time needed by an
algorithm for an input size “n”.
- Avarage-Case time complexity
is the execution of an algorithm having typical input of size “n”.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Notation for writting algorithm
14
In our DS(2) course we will express all the algorithm by
Pseudocodes representation.
This means that the algorithms are language and machine
independent.
The Psuedocode hides the implementation details and
focuses on the computational aspects of an algorithm
There are no grammar rules in the Pseudocode, it consists of
keywords and English. Like phrase which specify the flow of
control.
Pseudocode is not case sensitive.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
15
Each instruction takes only one unit of time.
a=a+1 one unit time
a=x^2+3*x-4 one unit time
Algorithm ABC(x,y)
cost time
SET Sum x+y c1 1
SET Mul x*y c2 1
Total Cost = c1 *1 + c2 * 1
= c1 + c2
T(n) = Cn0
The complexity of this algorithm is Θ(n0) Θ(1)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
16
Find the running for the following codes :
cost time
Sum 0 c1 1
Sum Sum + next c2 1
T(n)= c1 + c2 = c3
-------------------------------------------------------------------------------------
cost time
S 0 c1 1
For i 2 to n Do c2 n
S s+I c3 n-1
End For
S S/3 c4 1
T(n)= c1 + n c2 + (n-1) c3 + c4 T(n) is Θ(n)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
17
cost time
For i 1 to n Do c1 n+1
sum sum + 1 c2 n
End For
T(n) = (n+1) c1 + n c2 = (c1+ c2)n + c2
= c3n + c1
T(n) is Θ(n)
OR T(n) = =n
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
18
cost time
For i 1 To n Do c1 n+1
For j 1 To m Do c2 n(m+1)
anything() c3 mn
End For
End For
T(n)= (n+1) c1 + (n)(m+1) c2 + (n)(m)c3
= n c1 + c1 + mn c2 + n c2 + mn c3
T(n) is Θ(mn)
OR = = nm
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
19
For j 1 To n Do
For k 1 To j Do
anything()
End For
End For
T(n)= = =
T(n) is Θ(n2)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
20
For i 1 To n Do
For j i To n Do
anything()
End For
End For
T(n)= = = - +
= n2 - +n
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
21
Suppose that the function Bar(n) requires Θ(n2) what is the
complexity at the following algorithm ?
Algorithm Foo(n) cost time
Set x 0
For i 1 To n Do
Set x x + Bar(i)
End For
T(n) =
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
22
Algorithm Foo(n)
Set s 0 + =
Repeat Until n=1 Do
s s+1 (n-1) + (n-1) =
n n-1 2n-2
End Repeat
For i 1 To s Do T(n) is Θ(n)
anything()
End For
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Time Complexity
23
cost time
Algorithm Foo(n)
Set s 0
Repeat Until n=1 Do
s s+1
n n/2
End Repeat
For i 1 To s Do
anything()
End For
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Insertion Sort
24
Sort Array A using the Insertion-Sort algorithm
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Insertion Sort
25
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Insertion Sort
26
INSERTION-SORT (A, n) ⊳A[1 . . n]
cost time
For j ←2 to n do c1 n
Set key ←A[ j] c2 n-1
Set i ←j –1 c3 n-1
Repeat While (i > 0 and A[i] > key) do c4 tj
Set A[i+1] ←A[i] c5 tj-1
Set i ←i –1 c6 tj-1
End Repeat
A[i+1] = key c7 n-1
End For
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Insertion Sort
27
The running time of I-S algorithm is
∑(cost of step) * (number of times step is executed)
So
T(n)= n c1 + (n-1) c2 + (n-1)c3 + tj c4 + tj-1 c5 + tj-1 c6
+ (n-1) c7
* To solve the above T(n) , we should know firstly the value of
tj , and this is vary depends on the input.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Insertion Sort
28
The Best-Case
The array is already sorted
tj = 1 = (n-1)
T(n) = nc1 + nc2 – c2 + nc3 – c3 + nc4 – c4 + nc7 – c7
= (c1 + c2 + c3 + c4 + c7 )n + (-c2-c3-c4- c7)
= An+B
T(n) is Θ(n)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Insertion Sort
29
The Worse-Case
The array is in reverse stored order
tj = j
T(n)= n c1 + (n-1) c2 + (n-1)c3 + c4 + c5 + c6 + (n-1) c7
= -1 =
T(n) is Θ(n2)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Homework
30
Consider sorting n-numbers sorted in array A by first
finding the smallest element of A and putting it in the first
entry of another array B , then finding the second element
of A and putting it in the second entry of B . Then continue
in this manner for the n elements of B.
1. Write Pseudocode for this algorithm.
2. Compute the best and worse-case running time
3. Express these two running time using Θ-notation.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Asymptotic Notations
31
These notations are used to express the complexity
of a given algorithm, or used to compute two
algorithms or more that are designed to solve the
same problem.
1) Big-Oh Ο()
2) Big-Omega Ω()
3) Bog-Theta Θ()
4) Little-Oh o ()
5) Little-Omega ω()
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Asymptotic Notations
32
Big-Oh
- Upper-Bound
- Express the worse-case
Ο(g(n))={T(n):∃n0,c are positives, such that T(n) ≤ cg(n), ∀ 0}
∃ ∀n≥n
T(n)=2n+1 Є O(n)
T(n)=10n Є O(n)
T(n)=0.2n-7 Є O(n)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Asymptotic Notations
33
Prove that
T(n)= 2n+1 is O(n)
T(n) ≤ cg(n)
2n+1 ≤ cn
2n+1 ≤ 2n+n ∀ n ≥1
2n+1 ≤ 3n where c=3,n0=1
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Asymptotic Notations
34
Big-Omega
- Lower-Bound
- Express the best-case
Ω(g(n))={T(n):∃n0,c are positives, such that T(n) ≥ cg(n), ∀n ≥ n0}
∃ ∀
Big-Theta
- Tight Bound
- Express the best-case , worst-case, Avg-case
Θ(g(n))={T(n):∃n0,c1,c2 are positives, such that
c1g(n) ≥ T(n) ≥ c2g(n), ∀n ≥ n0}
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Asymptotic Notations
35
T/F
- T(n)= 3n2+4 is
1. T(n) Є O(n2)
2. T(n) Є O(n logn)
3. T(n) Є O(n5)
4. T(n) Є Ω(n2)
5. T(n) Є Ω(n5)
6. T(n) Є Ω(n logn)
7. T(n) Є Θ(n2)
8. T(n) Є Θ(n logn)
9. T(n) Є Θ(n5)
10. T(n) Є Θ(2n)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Divide and Conquer
36
E.g : 1,1,2,3,5,8,13,… ► The Fibonacci Series
- Incremental
- Recursive
Set F1 1
Set F2 1
FOR i 3 To n Do
Set F F1 + F2
Display F
Set F1 F2 Incremental Approach
Set F2 F
End For
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Divide and Conquer
37
Function fib(n)
{
If (n 1 OR n 0)
Return 1
Else
Call fib(n-1)+fib(n-2)
Recursive Approach
}
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Merge Sort
38
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Merge Sort
39
Divide:
Divide the problem into a number of subproblems that are themselves smaller
instances of the same type of problem.
Conquer:
Recursively solving these subproblems. If the subproblems are small enough,
solve them straightforward.
Combine:
Combine the solutions to the subproblems into the solution of original problem.
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Merge Sort
40
The running time of D-C algorithm can be
expressed as :
Θ(1) if n ≤ C
T(n)
a T( ) + D(n) + C(n)
a : # of sub problem : size of each subproblem
D(n) : Time to divide C(n): Time to combine
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Merge Sort
41
The running time for the merge-sort algorithm is :
Θ(1) n=1
T(n)
2 T( ) + Θ(n) n>1
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Merge Sort
42
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Master Method
43
How to solve the recurrence equations ?
There are many methods to solve the recurrence :
1. Master Method
2. Iteration Method
3. Recurrence Method
4. Substitution Method
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Master Method
44
T(n) = a T( )+f(n) where ≥1 , b >1 and
d= , then
Θ(nd) , if f(n) < nd
T(n) Θ(nd ) , f(n) = nd
Θ(f(n)) , f(n) > nd and
f( ) ≤ c f(n) where c<1
Regularity condition
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Master Method
45
Find the complexity of the merge-sort algorithm
using the master method
2 T( ) + Θ(n)
=2,b=2,d= =1
- Compare f(n) with nd
n=n f(n) = nd
- case #2 is satisfied
T(n) is Θ (n )
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Master Method
46
T(n) = 9 T( ) + n
=9,b=3,d= =2
- Compare f(n) with nd
n < n2 f(n) < nd
- case #1 is satisfied
T(n) is Θ(n2)
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Master Method
47
T(n) = T ( ) +1
=1,b= ,d= =0
- Compare f(n) with nd
1 = n0 f(n) = nd
- case #2 is satisfied
T(n) is Θ ( )
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Master Method
48
T(n) = 2T( ) +
=2 , b=5 , d = = 0.43
- Compare f(n) with nd
n0.5 > n 0.43
- Test 1 of case #3 is satisfied , we should check regularity
condition
2 f( ) ≤ c f(n)
2 ≤c
≤ c = 0.89
T(n) is Θ( )
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Homework
49
T(n) = 2 T( )+ n
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Thank you
50
This course DS2 of lectures by Dr. Hassan Alserhan
from BAU
Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com