Upcoming SlideShare
×

# 01 analysis-of-algorithms

504 views

Published on

Published in: Education
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
504
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
22
0
Likes
0
Embeds 0
No embeds

No notes for slide

### 01 analysis-of-algorithms

1. 1. Algoritm (CSC 206, Autumn’06)• Text (Required): Introduction to Algorithms by Cormen, Lieserson, Rivest & Stein• Instructor: Dr. Ali Sabbir• Office Rm 5009 D.
2. 2. Grading• Attendance is required• At least 1 midterm.• Exactly 1 in class Final.• Several quizzes (Announced and Unannounced)• Lots of assignments.
3. 3. Pre-Requisites• A solid math background.• Some exposure to high level language coding, e.g., C, C++ etc.• Preferred language for this course C++.
4. 4. Algorithms LECTURE 1 Analysis of Algorithms • Insertion sort • Asymptotic analysis • Merge sort • Recurrences
5. 5. Analysis of algorithmsThe theoretical study of computer-programperformance and resource usage.What’s more important than performance? • modularity • user-friendliness • correctness • programmer time • maintainability • simplicity • functionality • extensibility • robustness • reliability
6. 6. Why study algorithms and performance?• Algorithms help us to understand scalability.• Performance often draws the line between what is feasible and what is impossible.• Algorithmic mathematics provides a language for talking about program behavior.• Performance is the currency of computing.• The lessons of program performance generalize to other computing resources.• Speed is fun!
7. 7. The problem of sortingInput: sequence a1, a2, …, an of numbers.Output: permutation a1, a2, …, an suchthat a1  a2 …  an . Example: Input: 8 2 4 9 3 6 Output: 2 3 4 6 8 9
8. 8. Insertion sort INSERTION-SORT (A, n) A[1 . . n] for j ← 2 to n do key ← A[ j] i←j–1“pseudocode” while i > 0 and A[i] > key do A[i+1] ← A[i] i←i–1 A[i+1] = key
9. 9. Insertion sort INSERTION-SORT (A, n) A[1 . . n] for j ← 2 to n do key ← A[ j] i←j–1“pseudocode” while i > 0 and A[i] > key do A[i+1] ← A[i] i←i–1 A[i+1] = key 1 i j n A: key sorted
10. 10. Example of insertion sort 8 2 4 9 3 6
11. 11. Example of insertion sort 8 2 4 9 3 6
12. 12. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6
13. 13. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6
14. 14. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6
15. 15. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6
16. 16. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 4 8 9 3 6
17. 17. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 4 8 9 3 6
18. 18. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 4 8 9 3 6 2 3 4 8 9 6
19. 19. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 4 8 9 3 6 2 3 4 8 9 6
20. 20. Example of insertion sort 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 4 8 9 3 6 2 3 4 8 9 6 2 3 4 6 8 9 done
21. 21. Running time• The running time depends on the input: an already sorted sequence is easier to sort.• Parameterize the running time by the size of the input, since short sequences are easier to sort than long ones.• Generally, we seek upper bounds on the running time, because everybody likes a guarantee.
22. 22. Kinds of analysesWorst-case: (usually) • T(n) = maximum time of algorithm on any input of size n.Average-case: (sometimes) • T(n) = expected time of algorithm over all inputs of size n. • Need assumption of statistical distribution of inputs.Best-case: (bogus) • Cheat with a slow algorithm that works fast on some input.
23. 23. Machine-independent timeWhat is insertion sort’s worst-case time?• It depends on the speed of our computer: • relative speed (on the same machine), • absolute speed (on different machines).BIG IDEA:• Ignore machine-dependent constants.• Look at growth of T(n) as n → ∞ . “Asymptotic Analysis”
24. 24. -notationMath: (g(n)) = { f (n) : there exist positive constants c1, c2, and n0 such that 0  c1 g(n)  f (n)  c2 g(n) for all n  n0 }Engineering:• Drop low-order terms; ignore leading constants.• Example: 3n3 + 90n2 – 5n + 6046 = (n3)
25. 25. Asymptotic performance When n gets large enough, a (n2) algorithm always beats a (n3) algorithm. • We shouldn’t ignore asymptotically slower algorithms, however. • Real-world design situations often call for aT(n) careful balancing of engineering objectives. • Asymptotic analysis is a useful tool to help to n n0 structure our thinking.
26. 26. Insertion sort analysisWorst case: Input reverse sorted. n T ( n)   ( j )  n 2  [arithmetic series] j 2Average case: All permutations equally likely. n T ( n)   ( j / 2)  n 2  j 2Is insertion sort a fast sorting algorithm?• Moderately so, for small n.• Not at all, for large n.
27. 27. Merge sortMERGE-SORT A[1 . . n] 1. If n = 1, done. 2. Recursively sort A[ 1 . . n/2 ] and A[ n/2+1 . . n ] . 3. “Merge” the 2 sorted lists. Key subroutine: MERGE
28. 28. Merging two sorted arrays20 1213 117 92 1
29. 29. Merging two sorted arrays20 1213 117 92 1 1
30. 30. Merging two sorted arrays20 12 20 1213 11 13 117 9 7 92 1 2 1
31. 31. Merging two sorted arrays20 12 20 1213 11 13 117 9 7 92 1 2 1 2
32. 32. Merging two sorted arrays20 12 20 12 20 1213 11 13 11 13 117 9 7 9 7 92 1 2 1 2
33. 33. Merging two sorted arrays20 12 20 12 20 1213 11 13 11 13 117 9 7 9 7 92 1 2 1 2 7
34. 34. Merging two sorted arrays20 12 20 12 20 12 20 1213 11 13 11 13 11 13 117 9 7 9 7 9 92 1 2 1 2 7
35. 35. Merging two sorted arrays20 12 20 12 20 12 20 1213 11 13 11 13 11 13 117 9 7 9 7 9 92 1 2 1 2 7 9
36. 36. Merging two sorted arrays20 12 20 12 20 12 20 12 20 1213 11 13 11 13 11 13 11 13 117 9 7 9 7 9 92 1 2 1 2 7 9
37. 37. Merging two sorted arrays20 12 20 12 20 12 20 12 20 1213 11 13 11 13 11 13 11 13 117 9 7 9 7 9 92 1 2 1 2 7 9 11
38. 38. Merging two sorted arrays20 12 20 12 20 12 20 12 20 12 20 1213 11 13 11 13 11 13 11 13 11 137 9 7 9 7 9 92 1 2 1 2 7 9 11
39. 39. Merging two sorted arrays20 12 20 12 20 12 20 12 20 12 20 1213 11 13 11 13 11 13 11 13 11 137 9 7 9 7 9 92 1 2 1 2 7 9 11 12
40. 40. Merging two sorted arrays20 12 20 12 20 12 20 12 20 12 20 1213 11 13 11 13 11 13 11 13 11 137 9 7 9 7 9 92 1 2 1 2 7 9 11 12 Time = (n) to merge a total of n elements (linear time).
41. 41. Analyzing merge sort T(n) MERGE-SORT A[1 . . n] (1) 1. If n = 1, done. 2T(n/2) 2. Recursively sort A[ 1 . . n/2 ]Abuse and A[ n/2+1 . . n ] . (n) 3. “Merge” the 2 sorted lists Sloppiness: Should be T( n/2 ) + T( n/2 ) , but it turns out not to matter asymptotically.
42. 42. Recurrence for merge sort (1) if n = 1; T(n) = 2T(n/2) + (n) if n > 1.• We shall usually omit stating the base case when T(n) = (1) for sufficiently small n, but only when it has no effect on the asymptotic solution to the recurrence.• CLRS and Lecture 2 provide several ways to find a good upper bound on T(n).
43. 43. Recursion treeSolve T(n) = 2T(n/2) + cn, where c > 0 is constant.
44. 44. Recursion treeSolve T(n) = 2T(n/2) + cn, where c > 0 is constant. T(n)
45. 45. Recursion treeSolve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn T(n/2) T(n/2)
46. 46. Recursion treeSolve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn/2 cn/2 T(n/4) T(n/4) T(n/4) T(n/4)
47. 47. Recursion treeSolve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn/2 cn/2 cn/4 cn/4 cn/4 cn/4 (1)
48. 48. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn/2 cn/2h = lg n cn/4 cn/4 cn/4 cn/4 (1)
49. 49. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn cn/2 cn/2h = lg n cn/4 cn/4 cn/4 cn/4 (1)
50. 50. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn cn/2 cn/2 cnh = lg n cn/4 cn/4 cn/4 cn/4 (1)
51. 51. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn cn/2 cn/2 cnh = lg n cn/4 cn/4 cn/4 cn/4 cn … (1)
52. 52. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn cn/2 cn/2 cnh = lg n cn/4 cn/4 cn/4 cn/4 cn … (1) #leaves = n (n)
53. 53. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn cn/2 cn/2 cnh = lg n cn/4 cn/4 cn/4 cn/4 cn … (1) #leaves = n (n) Total (n lg n)
54. 54. Conclusions• (n lg n) grows more slowly than (n2).• Therefore, merge sort asymptotically beats insertion sort in the worst case.• In practice, merge sort beats insertion sort for n > 30 or so.• Go test it out for yourself!