### SlideShare for iOS

by Linkedin Corporation

FREE - On the App Store

- Total Views
- 798
- Views on SlideShare
- 798
- Embed Views

- Likes
- 1
- Downloads
- 26
- Comments
- 0

No embeds

Uploaded via SlideShare as Microsoft PowerPoint

© All Rights Reserved

- 1. Data Structures and Algorithms Lecture 1: Asymptotic Notations
- 2. Basic Terminologies • Algorithm – Outline – Essence of a computational procedure – Step by step instructions • Program – Implementation of an Algorithm in some programming language • Data Structure – Organization of Data needed to solve the problem effectively – Data Structure you already know: Array
- 3. Algorithmic Problem Specification of Input ? Specification of Output as a function of Input • Infinite number of input instances satisfying the specification • E.g.: Specification of Input: • A sorted non-decreasing sequence of natural numbers of non-zero, finite length: • 1, 20,908,909,1000,10000,1000000 • 3
- 4. Algorithmic Solution Input instance adhering to Specification Algorithm Output related to the Input as required • Algorithm describes the actions on the input instances • Infinitely many correct algorithms may exist for the same algorithmic problem
- 5. Characteristics of a Good Algorithm • Efficient – Small Running Time – Less Memory Usage • Efficiency as a function of input size – The number of bits in a data number – The number of data elements
- 6. Measuring the running time • Experimental Study – Write a program that implements the algorithm – Run the program with data sets of varying size and composition – Use system defined functions like clock() to get an measurement of the actual running time
- 7. Measuring the running time • Limitations of Experimental Study – It is necessary to implement and test the algorithm in order to determine its running time. – Experiments can be done on a limited set of inputs, and may not be indicative of the running time on other inputs not included in the experiment – In order to compare 2 algorithms same hardware and software environments must be used.
- 8. Beyond Experimental Study • We will develop a general methodology for analyzing the running time of algorithms. This approach – Uses a high level description of the algorithm instead of testing one of its implementations – Takes into account all possible inputs – Allows one to evaluate the efficiency of any algorithm in a way that is independent of the hardware and software environment.
- 9. Pseudo - Code • A mixture of natural language and high level programming concepts that describes the main ideas behind a generic implementation of a data structure or algorithm. • E.g. Algorithm arrayMax(A,n) – Input: An array A storing n integers – Output: The maximum element in A. currMax A[0] for i 1 to n-1 do if currMax < A[i] then currMax return currMax A[i]
- 10. Pseudo - Code • It is more structured that usual prose but less formal than a programming language • Expressions: – Use standard mathematical symbols to describe numeric and Boolean expressions – Use ‘ ’ for assignment instead of “=” – Use ‘=’ for equality relationship instead of “==” • Function Declarations: – Algorithm name (param1 , param2)
- 11. Pseudo - Code • Programming Constructs: – – – – – Decision structures: if…then….[else….] While loops: while…..do….. Repeat loops: Repeat………until……. For loop: for……do……….. Array indexing: A[i], A[I,j] • Functions: – Calls: return_type function_name(param1 ,param2) – Returns: return value
- 12. Analysis of Algorithms • Primitive Operation: – Low level operation independent of programming language. – Can be identified in a pseudo code. • For e.g. – Data Movement (assign) – Control (branch, subroutine call, return) – Arithmetic and Logical operations (e.g. addition comparison) – By inspecting the pseudo code, we can count the number of primitive operations executed by an algorithm.
- 13. Example: Sorting OUPUT Permutation of the sequence of numbers in non-decreasing order INPUT Sequence of numbers a1,a2,a3,a4,….an 2, 5 , 4, 10, 7 Sort Correctness (Requirements for the output) For any given input the algorithm halts with the output: • b1 < b2 < b3 < b4 < ….. < bn • b1, b2, b3, b4,…. bn is a permutation of a1, a2, a3, a4, …. an b1,b2,b3,b4,….bn 2, 4 , 5, 7, 10 Running Time depends on • No. of elements (n) • How sorted (partial) the numbers are? • Algorithm
- 14. Insertion Sort • Used while playing cards 3 4 6 8 9 A 1 i j Strategy • Start “empty handed” • Insert a card in the right position of the already sorted hand • Continue until all cards are inserted or sorted 7 2 5 1 n INPUT: An array of Integers A[1..n] OUTPUT: A permutation of A such that A[1] <= A[2] <= A[3]<= ..<=A[n] For j 2 to n do key A[j] Insert A*j+ into sorted sequence A*1…j-1] i j-1 while i >0 and A[i] >key do A[i+1] A[i] i- A[i+1] key
- 15. Analysis of Insertion Sort • For j 2 to n do • key A[j] • Insert A[j] into sorted sequence A*1…j-1] • i j-1 • while i >0 and A[i] >key • do A[i+1] A[i] • i- • A[i+1] key Total Time = n(c1 + c2 + c3 + c7) + Cost c1 c2 Times n n-1 c3 c4 c5 c6 c7 n-1 n-1 (c4 + c5 + c6)–( c2 +c3 +c5+c6+c7) tj counts the number of times the values have to be shifted in one iteration
- 16. Best / Average/ Worst Case: Difference made by tj • Total Time = n(c1 + c2 + c3 + c7) + +c3 +c5+c6+c7) • Best Case (c4 + c5 + c6)–( c2 – Elements already sorted – tj = 1 – Running time = f(n) i.e. Linear Time • Worst Case – Elements are sorted in inverse order – tj = j – Running time = f(n2) i.e. Quadratic Time • Average Case – tj = j/2 – Running Time = f(n2) i.e. Quadratic Time
- 17. Best / Average/ Worst Case • For a Specific input size say n Running Time (sec) 5 Worst Case 4 Average Case 3 Best Case 2 1 A B C D E F G H I J K L M N Input instance
- 18. Best / Average/ Worst Case • Varying input size Worst case Running Time (sec) 50 Average Case 40 Best Case 30 20 10 10 20 30 40 Input Size 50 60
- 19. Best / Average/ Worst Case • Worst Case: – Mostly used – It is an Upper bound of how bad a system can be. – In cases like surgery or air traffic control knowing the worst case complexity is of crucial importance. • Average case is often as bad as worst case. • Finding an average case can be very difficult.
- 20. Asymptotic Analysis • Goal: Simplify analysis of running time by getting rid of details which may be affected by specific implementation and hardware – Like rounding 1000001 to 1000000 – 3n2 to n2 • Capturing the essence: How the running time of an algorithm increases with the size of the input in the limit – Asymptotically more efficient algorithms are best for all but small inputs.
- 21. Asymptotic Notation • The “big Oh” (O) Notation – Asymptotic Upper Bound – f(n) = O(g(n)), if there exists constants c and n0 such that f(n) ≤ cg(n) for n ≥ n0 – f(n) and g(n) are functions over non negative nondecreasing integers • Used for worst case analysis c. g(n) f(n) n0 n f(n) = O(g(n))
- 22. Examples • E.g. 1: – – – – – f(n) = 2n + 6 g(n) = n c =4 n0 = 3 Thus, f(n) = O(n) • E.g. 2: – f(n) = n2 – g(n) = n – f(n) is not O(n) because there exists no constants c and n0 such that f(n) ≤ cg(n) for n ≥ n0 • E.g. 3: – – – – – f(n) = n2 + 5 g(n) = n2 c=2 n0 = 2 Thus, f(n) = O(n2)
- 23. Asymptotic Notation • Simple Rules: – Drop lower order terms and constant factors • 50 nlogn is O(nlogn) • 7n – 3 is O(n) • 8n2logn + 5n2 + n is O(n2logn) – Note that 50nlogn is also O(n5) but we will consider it to be O(nlogn) it is expected that the approximation should be of as lower value as possible.
- 24. Comparing Asymptotic Analysis of Running Time • Hierarchy of f(n) – Log n < n < n2 < n3 < 2n • Caution!!!!! – An algorithm with running time of 1,000,000 n is still O(n) but might be less efficient than one running in time 2n2, which is O(n2) when n is not very large.
- 25. Examples of Asymptotic Analysis • Algorithm: prefixAvg1(X) • Input: An n element array X of numbers • Output: An n element array A of numbers such that A[i] is the average of elements X[0]…X[i]. for i 0 to n-1 do a 0 for j 0 to i do a a+X[j] A[i] a/(i+1) return A n iterations 1 Step i iterations i = 0 , 1, …., i-1 • Analysis: Running Time O(n2) roughly.
- 26. Examples of Asymptotic Analysis • Algorithm: prefixAvg2(X) • Input: An n element array X of numbers • Output: An n element array A of numbers such that A[i] is the average of elements X[0]…X[i]. s 0 for i 0 to n do s s + X[i] A[i] s/(i+1) return A • Analysis: Running time is O(n)
- 27. Asymptotic Notation (Terminologies) • Special cases of Algorithms – Logarithmic : O(log n) – Linear: O(n) – Quadratic: O(n2) – Polynomial: O(nk), k ≥ 1 – Exponential: O(an), a>1 • “Relatives” of Big Oh – Big Omega (Ω(f(n)) : Asymptotic Lower Bound – Big Theta (Θ(f(n)): Asymptotic Tight Bound
- 28. Asymptotic Notation • The big Omega (Ω) Notation – Asymptotic Lower bound – f(n) = Ω(g(n), if there exists constants c and n0 such that c.g(n) ≤ f(n) for n > n0 • Used to describe best case running times or lower bounds of asymptotic problems – E.g: Lower bound of searching in an unsorted array is Ω(n).
- 29. Asymptotic Notation • The Big Theta (Θ) notation – Asymptotically tight bound – f(n) = Θ(g(n)), if there exists constants c1, c2 and n0 such that c1.g(n) ≤ f(n) ≤ c2.g(n) for n > n0 – f(n) = Θ(g(n)), iff f(n) = O(g(n)) and f(n) = Ω(g(n), – O(f(n)) is often misused instead of Θ(f(n))
- 30. Asymptotic Notation • There are 2 more notations – Little oh notation (o), f(n) = o(g(n)) • For every c there should exist a no. n0 such that f(n) ≤ o(g(n) for n > n0. – Little Omega notation (ω) • Analogy with real numbers – – – – – f(n) = O(g(n)) , f(n) = Ω(g(n)), f(n) = Θ(g(n)), f(n) = o(g(n)), f(n) = ω(g(n)), f≤g f≥g f=g f<g f>g • Abuse of Notation: – f(n) = O(g(n)) actually means f(n) є O(g(n))
- 31. Comparison of Running times
- 32. Assignment • Write a C program to implement i) Insertion sort and ii) Bubble Sort. • Perform an Experimental Study and graphically show the time required for both the program to run. • Consider varied size of inputs and best case and worst case for each of the input sets.

Full NameComment goes here.Mikael Bourges-Sevenier, Director at Aptina Imaging 6 days ago