Introduction

1. Introduction to Data Structure and
Algorithms
Manoj Kumar Rana
Research Asst. Professor
Dept. of Computing Technologies
SRM Institute of Science and Technology

2. Data structure and Algorithms
• Algorithm
▫ A step-by-step timely instruction or outline of a
computational procedure
• Program
▫ Implementation of an algorithm by using a
programming language
• Data Structure
▫ Organization of data needed to solve a problem
2

3. Data structure and Algorithms (2)
• Object
▫ An instance of a set of heterogeneous data, defining
behavior or character of a real-life entity (e.g. A Car
object which contains data like color, size price, etc.)
• Object oriented program
▫ A program where objects can interact with each other
to ease the implementation of the algorithm
• Relation between data structure and object
▫ Object is also one data type.
▫ Multiple objects can be organized by following a
specific data structure
3

4. Algorithmic problem
• Infinite set of input instances satisfying the
specification. For example, a sorted
non-increasing sequence of numbers of finite
length:
▫ 100000, 20000, 1000, 900, 20, 1.
Specification
of input
Specification of
output as a
function of
input
?
4

5. Algorithmic solution
• Algorithm describes the action on input instance
• Infinitely many correct algorithms for a
particular problem
Input instance
adhering to the
specification
Output as a
function of
input
Algorithm
5

6. Criteria of a good algorithm
• Efficient
▫ Running time
▫ Space taken
• Efficiency is measured as a function of input size
▫ Number of bits used to represent input data
element
▫ Number of data elements
6

7. Measuring the running time
• How could we measure the running time of an
algorithm?
▫ Experimental study
● Write the program that implements the algorithm
● Run the program with various data sizes and types
● Use the function System.CurrentTimeInMillis
() to get the accurate running time of the program
7

8. Limitation of experimental study
• It is required to implement and test the
algorithm in order to determine its running
time.
• Experiment is always done on limited set of
inputs, not done on other inputs.
• This study depends on the hardware and
software used in the experiment. To compare
running times of two algorithms, same hardware
and software environments must be used.
8

9. Beyond experimental study
• We will develop a general methodology for
analyzing running time of algorithms
▫ Uses a high-level description of the algorithm,
instead of using one of its implementation
▫ Takes into account all possible inputs
▫ Independent of hardware and software
environment
9

10. Pseudo code
• A mixture of natural languages and high-level
programming concepts that describes the main
idea behind a generic implementation of an
algorithm or data structure
▫ Eg:ArrayMax (A, n)
Input: Array A and number of elements in A, n
Output: Maximum element in A
CurrMax A[1]
for i 2 to n do
if CurrMax < A[i] then CurrMax A[i]
return CurrMax
10

11. Analysis of Algorithm
• Primitive Operation: Low-level operations in
pseudo code, independent of programming
languages
▫ data movement (assign)
▫ control (branch, sub-routine call, return, etc.)
▫ arithmetic and logical operations (addition,
comparison, etc.)
• By inspecting pseudo code, we can count
number of primitive operations executed by an
algorithm
11

12. Example: Sorting
Sort
Input
Sequence of numbers
Output
A permutation of the
sequence of numbers
a1
,a2
,a3
, ………., an
b1
,b2
,b3
, ………., bn
Correctness (Requirements for the
output)
For any given input the algorithm
halts with the following output:
b1
<b2
<b3
< ………… < bn
b1
, b2
, b3
, ……….., bn
is the
permutation of a1
, a2
, a3
, ……., an
Running time depends on
number of elements
how partially sorted they are
the algorithm
12

13. Insertion Sort
4 7 2 10 7 3 8
1 n
i
j
Strategy:
Insert an element in the right
position of a set of already
sorted elements
Continue until all the
elements are sorted
Input: A[1…n], an array of integers
Output: a permutation of A, such that
A[1]≤A[2] ≤A[3] ≤……………. ≤ A[n]
for j 2 to n do
key A[j]
Insert key into the sorted sequence A[1, j-1]
i j-1
while i > 0 and A[i] > key
do A[i+1] A[i]
i--
A[i+1] key
13

14. Analysis of Insertion Sort
primitive operations cost times
for j 2 to n do c1
n
key A[j] c2
n-1
Insert key into the sorted sequence A[1, j-1] 0 n-1
i j-1 c3
n-1
while i > 0 and A[i] > key c4
do A[i+1] A[i] c5
i-- c6
A[i+1] key c7
n-1
Total time = n (c1
+ c2
+ c3
– c5
– c6
+ c7
) + (c4
+
c5
+c6
) - (c2
+c3
+ c7
)
14

15. Best, worst and Average cases
• Best case: elements already sorted, tj
= 1,
running time = f(n) i.e. linear time.
• Worst case: elements are in decreasing order,
tj
= j, running time = f(n2
) i.e. quadratic time
• Average case: tj
= j/2, running time = f(n2
), i.e.
quadratic time
Total time = n (c1
+ c2
+ c3
– c5
– c6
+ c7
) + (c4
+ c5
+c6
) - (c2
+c3
+ c7
)
15

16. Best/worst/average case
For a specific input size n, investigate running
times of different instances
Running
time
(min)
Input instances
best case
worst case
average case
16

17. Asymptotic Analysis
• Goal: simplifying analysis of running time by
getting rid of “details” which may be affected by
specific implementation and hardware
▫ like, rounding 10,000.01 ≈ 10,000
▫ 6n2
≈ n2
• How the running time of an algorithm increases
with the size of the input in a limit
▫ asymptotically more efficient algorithms are best
for all but small inputs
17

18. Asymptotic notation
• The “big-oh” O-notation
▫ asymptotic upper bound
▫ f(n) = O(g(n)), if there
exists constants c and n0
,
s.t. f(n) ≤ cg(n) for n ≥ n0
▫ f(n) and g(n) are functions
over non-negative integers
• Used for worst-case
analysis
cg(n)
f(n)
Input size
Running
time
n0
18

19. Asymptotic notation
• n2
is not O(n) because there is no c and n0
s.t.:
n2
≤ cn for n ≥ n0
▫ no matter how large a c is chosen there exist an n big
enough s.t. n2
> cn
• Simple rule: drop lower order terms and constant
factors
▫ 40 n log n is O(n log n)
▫ 3n-7 is O(n)
▫ 8n2
log n +n2
+ 7n +6 is O(n2
log n)
• Note: although (40 n2
) is O(n5
), it is expected that
such an approximation be as small an order as
possible
19

20. Asymptotic analysis of running time
• The O-notation expresses number of primitive
operations executed as function of input size
• Comparing asymptotic running times
▫ an algorithm running in O(n) time is better than an
algorithm running in O(n2
) time
▫ Similarly O(log n) is better than O(n)
▫ hierarchy: log n < n < n2
< n3
< 2n
• Caution! Beware of very large constant factor. An
algorithm running in 10000000 n time is still O(n),
but might be less efficient than an algorithm running
in time 2n2
time, which is O(n2
)
20

21. Example of Asymptotic Analysis
Algorithm prefixAverages1 (X)
Input: An n element array X of integer numbers
Output: An n element array A of integer numbers
where A[i] is the average of elements X[1], X[2],
……, X[i]
for i 1 to n do
a 0
for j 1 to i do
a a + X[j]
A[i] a/i
return A
Analysis: running time is O(n2
)
i iterations
with i = 1, 2,
3, ….., n
n iterations
21

22. A better Algorithm
Algorithm prefixAverages2 (X)
Input: An n element array X of integer numbers
Output: An n element array A of integer numbers
where A[i] is the average of elements X[1], X[2], ……,
X[i]
m 0
for i 1 to n do
m m + X[i]
A[i] s/i
return A
Analysis: running time is O(n)
22

23. Asymptotic Notation (terminology)
• Special classes of algorithms:
▫ Logarithmic: O(log n)
▫ Linear: O(n)
▫ Quadratic: O(n2
)
▫ Polynomial: O(nk
), k ≥ 1
▫ Exponential: O(an
), a > 1
• Abuse of notation: f(n) = O(g(n)) actually means
f(n) ε O(g(n))
23

24. Exercises
1.
2.
3. f(n) = Log n!, f(n) ε O(?)
24

25. Thank You!
25