Advance Data Structure

Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com 1

Data Structure Advanced

Algorithm
2

Well-defined sequence of computational steps.
The word Algorithm refers to a precise method
useable by a computer for solution of a problem.
The Algorithm accepts a set of values as input
and produces a set of values as output.

Ramzi Shihadeh Alqrainy qcs_2008@yahoo.com

Examples
3

Algorithm written in Human language

Input a,b,c
Compute the sum of a,b,c
Divide the Sum by 3
Display the Result


Example
4

Algorithm written in Programming language.


Example
5

Algorithm written in Pseudocode Syntax.
Step 1: Promt a,b,c
Step 2: Let Sum a+b+c
Step 3: Let Avg Sum / 3
Step 4: Output Avg


ALGORITHM
6

What do we mean by input ?
The word input interdependent on the type of
problem.
- Set of values or elements
e.g in the array problem
- A Set of bits
e.g in the string problem
- A Set of objects
e.g in the graph problems the input may be the set
of vertices and edges.


The Properties of Algorithm
7

It must be correct
It must be composed of a series of concrete
steps.
There can be no ambiguity as to which step will
be performed next.
It must be terminate.


Analysis of Algorithm
8

What is a good algorithm ?
- Is a clear algorithm ??
- Has a minimum steps ??
- Decreases the CPU time
- Has a minimum memory (RAM) space ?
- It is a good mapping for the problem.
- Uses minimum spaces of disk I/O

** In this course the best Algorithm is the one that
uses the minimum CPU resources

Study of Algorithm
9

Design Analysis

Methods and technique which Mathematical comparison of
yield a good and useful algorithm ( without
Algorithm to solve a problem. actually implement )


Algorithms
10
Types , methods , and techniques of ALG :
- Incremental Algorithms
- Simple Recursive Algorithms
- Divide and Conquer Algorithms
- Greedy Algorithms
- Dynamic Algorithms
- Backtracking Algorithms
- Branch and Bound Algorithms
- Brute and Force Algorithms
- Randomized Algorithms


Analysis of Algorithm
11

The analysis of an algorithm focuses on the time
and space complexity.
The Time Complexity is the amount of time
required by an algorithm to run to completion.
The Time Complexity is a function of input size
“n” T(n)
The Time Complexity refers to the amount of time
required by an ALG to run completion


Time Complexity
12


Time Complexity
13

Different times can arise for the same algorithm
- Worse-Case time complexity
is a function defined by the maximum amount of time needed by an
algorithm for an input size “n”.
- Best-Case time complexity
is a function defined by the minimum amount of time needed by an
algorithm for an input size “n”.
- Avarage-Case time complexity
is the execution of an algorithm having typical input of size “n”.


Notation for writting algorithm
14

In our DS(2) course we will express all the algorithm by
Pseudocodes representation.
This means that the algorithms are language and machine
independent.
The Psuedocode hides the implementation details and
focuses on the computational aspects of an algorithm
There are no grammar rules in the Pseudocode, it consists of
keywords and English. Like phrase which specify the flow of
control.
Pseudocode is not case sensitive.


Time Complexity
15

Each instruction takes only one unit of time.
a=a+1 one unit time
a=x^2+3*x-4 one unit time
Algorithm ABC(x,y)
cost time
SET Sum x+y c1 1
SET Mul x*y c2 1
Total Cost = c1 *1 + c2 * 1
= c1 + c2
T(n) = Cn0
The complexity of this algorithm is Θ(n0) Θ(1)

Time Complexity
16

Find the running for the following codes :
cost time
Sum 0 c1 1
Sum Sum + next c2 1
T(n)= c1 + c2 = c3
-------------------------------------------------------------------------------------
cost time
S 0 c1 1
For i 2 to n Do c2 n
S s+I c3 n-1
End For
S S/3 c4 1
T(n)= c1 + n c2 + (n-1) c3 + c4 T(n) is Θ(n)

Time Complexity
17

cost time
For i 1 to n Do c1 n+1
sum sum + 1 c2 n
End For
T(n) = (n+1) c1 + n c2 = (c1+ c2)n + c2
= c3n + c1
T(n) is Θ(n)
OR T(n) = =n


Time Complexity
18
cost time
For i 1 To n Do c1 n+1
For j 1 To m Do c2 n(m+1)
anything() c3 mn
End For
End For
T(n)= (n+1) c1 + (n)(m+1) c2 + (n)(m)c3
= n c1 + c1 + mn c2 + n c2 + mn c3
T(n) is Θ(mn)
OR = = nm


Time Complexity
19

For j 1 To n Do
For k 1 To j Do
anything()
End For
End For

T(n)= = =

T(n) is Θ(n2)


Time Complexity
20

For i 1 To n Do
For j i To n Do
anything()
End For
End For
T(n)= = = - +

= n2 - +n


Time Complexity
21

Suppose that the function Bar(n) requires Θ(n2) what is the
complexity at the following algorithm ?

Algorithm Foo(n) cost time
Set x 0
For i 1 To n Do
Set x x + Bar(i)
End For
T(n) =


Time Complexity
22

Algorithm Foo(n)
Set s 0 + =
Repeat Until n=1 Do
s s+1 (n-1) + (n-1) =
n n-1 2n-2
End Repeat
For i 1 To s Do T(n) is Θ(n)
anything()
End For


Time Complexity
23
cost time
Algorithm Foo(n)
Set s 0
Repeat Until n=1 Do
s s+1
n n/2
End Repeat
For i 1 To s Do
anything()
End For


Insertion Sort
24

Sort Array A using the Insertion-Sort algorithm


Insertion Sort
25


Insertion Sort
26

INSERTION-SORT (A, n) ⊳A[1 . . n]
cost time
For j ←2 to n do c1 n
Set key ←A[ j] c2 n-1
Set i ←j –1 c3 n-1
Repeat While (i > 0 and A[i] > key) do c4 tj
Set A[i+1] ←A[i] c5 tj-1

Set i ←i –1 c6 tj-1
End Repeat
A[i+1] = key c7 n-1
End For

Insertion Sort
27

The running time of I-S algorithm is
∑(cost of step) * (number of times step is executed)
So
T(n)= n c1 + (n-1) c2 + (n-1)c3 + tj c4 + tj-1 c5 + tj-1 c6
+ (n-1) c7

* To solve the above T(n) , we should know firstly the value of
tj , and this is vary depends on the input.


Insertion Sort
28

The Best-Case

The array is already sorted

tj = 1 = (n-1)

T(n) = nc1 + nc2 – c2 + nc3 – c3 + nc4 – c4 + nc7 – c7
= (c1 + c2 + c3 + c4 + c7 )n + (-c2-c3-c4- c7)
= An+B
T(n) is Θ(n)

Insertion Sort
29

The Worse-Case
The array is in reverse stored order
tj = j
T(n)= n c1 + (n-1) c2 + (n-1)c3 + c4 + c5 + c6 + (n-1) c7

= -1 =

T(n) is Θ(n2)


Homework
30

Consider sorting n-numbers sorted in array A by first
finding the smallest element of A and putting it in the first
entry of another array B , then finding the second element
of A and putting it in the second entry of B . Then continue
in this manner for the n elements of B.

1. Write Pseudocode for this algorithm.
2. Compute the best and worse-case running time
3. Express these two running time using Θ-notation.


Asymptotic Notations
31

These notations are used to express the complexity
of a given algorithm, or used to compute two
algorithms or more that are designed to solve the
same problem.
1) Big-Oh Ο()
2) Big-Omega Ω()
3) Bog-Theta Θ()
4) Little-Oh o ()
5) Little-Omega ω()


32

Big-Oh
- Upper-Bound
- Express the worse-case

Ο(g(n))={T(n):∃n0,c are positives, such that T(n) ≤ cg(n), ∀ 0}
∃ ∀n≥n

T(n)=2n+1 Є O(n)
T(n)=10n Є O(n)
T(n)=0.2n-7 Є O(n)


33

Prove that
T(n)= 2n+1 is O(n)

T(n) ≤ cg(n)
2n+1 ≤ cn
2n+1 ≤ 2n+n ∀ n ≥1
2n+1 ≤ 3n where c=3,n0=1


34

Big-Omega
- Lower-Bound
- Express the best-case

Ω(g(n))={T(n):∃n0,c are positives, such that T(n) ≥ cg(n), ∀n ≥ n0}
∃ ∀
Big-Theta
- Tight Bound
- Express the best-case , worst-case, Avg-case
Θ(g(n))={T(n):∃n0,c1,c2 are positives, such that
c1g(n) ≥ T(n) ≥ c2g(n), ∀n ≥ n0}


35

T/F
- T(n)= 3n2+4 is
1. T(n) Є O(n2)
2. T(n) Є O(n logn)
3. T(n) Є O(n5)
4. T(n) Є Ω(n2)
5. T(n) Є Ω(n5)
6. T(n) Є Ω(n logn)
7. T(n) Є Θ(n2)
8. T(n) Є Θ(n logn)
9. T(n) Є Θ(n5)
10. T(n) Є Θ(2n)


Divide and Conquer
36

E.g : 1,1,2,3,5,8,13,… ► The Fibonacci Series
- Incremental
- Recursive
Set F1 1
Set F2 1
FOR i 3 To n Do
Set F F1 + F2
Display F
Set F1 F2 Incremental Approach
Set F2 F
End For


Divide and Conquer
37

Function fib(n)
{
If (n 1 OR n 0)
Return 1
Else
Call fib(n-1)+fib(n-2)
Recursive Approach
}


Merge Sort
38


Merge Sort
39

Divide:
Divide the problem into a number of subproblems that are themselves smaller
instances of the same type of problem.
Conquer:
Recursively solving these subproblems. If the subproblems are small enough,
solve them straightforward.
Combine:
Combine the solutions to the subproblems into the solution of original problem.


Merge Sort
40

The running time of D-C algorithm can be
expressed as :
Θ(1) if n ≤ C
T(n)
a T( ) + D(n) + C(n)
a : # of sub problem : size of each subproblem

D(n) : Time to divide C(n): Time to combine


Merge Sort
41

The running time for the merge-sort algorithm is :

Θ(1) n=1
T(n)
2 T( ) + Θ(n) n>1


Merge Sort
42


Master Method
43

How to solve the recurrence equations ?
There are many methods to solve the recurrence :

1. Master Method
2. Iteration Method
3. Recurrence Method
4. Substitution Method


Master Method
44

T(n) = a T( )+f(n) where ≥1 , b >1 and
d= , then

Θ(nd) , if f(n) < nd
T(n) Θ(nd ) , f(n) = nd
Θ(f(n)) , f(n) > nd and
f( ) ≤ c f(n) where c<1

Regularity condition


Master Method
45

Find the complexity of the merge-sort algorithm
using the master method
2 T( ) + Θ(n)
=2,b=2,d= =1
- Compare f(n) with nd
n=n f(n) = nd
- case #2 is satisfied
T(n) is Θ (n )


Master Method
46

T(n) = 9 T( ) + n

=9,b=3,d= =2
n < n2 f(n) < nd
T(n) is Θ(n2)


Master Method
47

T(n) = T ( ) +1

=1,b= ,d= =0
1 = n0 f(n) = nd
T(n) is Θ ( )


Master Method
48
T(n) = 2T( ) +
=2 , b=5 , d = = 0.43
n0.5 > n 0.43
- Test 1 of case #3 is satisfied , we should check regularity
condition
2 f( ) ≤ c f(n)

2 ≤c

≤ c = 0.89
T(n) is Θ( )


Homework
49

T(n) = 2 T( )+ n


Thank you
50

This course DS2 of lectures by Dr. Hassan Alserhan
from BAU


Advance Data Structure

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to Advance Data Structure

Similar to Advance Data Structure (20)

More from Ramzi Alqrainy

More from Ramzi Alqrainy (20)

Recently uploaded

Recently uploaded (20)

Advance Data Structure