Introduction to algorithms

379 views

Published on

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
379
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
25
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Introduction to algorithms

  1. 1. Algorithms Sandeep Kumar Poonia Head Of Dept. CS/IT B.E., M.Tech., UGC-NET LM-IAENG, LM-IACSIT,LM-CSTA, LM-AIRCC, LM-SCIEI, AM-UACEE
  2. 2. Algorithms Introduction Proof By Induction Asymptotic notation
  3. 3. Pre Computer Era: • In the pre computer era various models of computation were proposed. The models worked on a predefined set of operations. The primary focus in this era was on “What can be computed and what cannot be computed?” This field of study was called Computability Theory. Sandeep Kumar Poonia
  4. 4. Post Computer Era: • With ultimate computability in terms of Turing Machines, the primary focus in the post computer era was Complexity Theory. People were more interested to find out “How well can it be computed?” Sandeep Kumar Poonia
  5. 5. Sandeep Kumar Poonia
  6. 6. Sandeep Kumar Poonia
  7. 7. Sandeep Kumar Poonia
  8. 8. As a corollary to the above mentioned result we can easily see that a number n must be halved floor(log2n) + 1 times to reach 0. Sandeep Kumar Poonia
  9. 9. Algorithms •An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. •An algorithm is thus a sequence of computational steps that transform the input into the output. Sandeep Kumar Poonia
  10. 10. Algorithms Sandeep Kumar Poonia
  11. 11. Algorithms Sandeep Kumar Poonia
  12. 12. Properties of Algorithms Sandeep Kumar Poonia
  13. 13. Steps to develop an algorithm Sandeep Kumar Poonia
  14. 14. Algorithms as a technology •Suppose computers were infinitely fast and •computer memory was free. Would you have any reason to study algorithms?......YES(See an example) let us assume a faster computer (computer A) running insertion sort against a slower computer (computer B) running merge sort. They each must sort an array of one million numbers. Suppose that computer A executes one billion instructions per second and computer B executes only ten million instructions per second, so that computer A is 100 times faster than computer B in raw computing power. insertion sort, takes time roughly equal to c1n2 to sort n items, merge sort, takes time roughly equal to c2n lg n Sandeep Kumar Poonia
  15. 15. Algorithms as a technology To sort one million numbers, • Computer A takes • Computer B takes By using an algorithm whose running time grows more slowly, even with a poor compiler, computer B runs 20 times faster than computer A Sandeep Kumar Poonia
  16. 16. Review: Induction • Suppose  S(k) is true for fixed constant k o Often k = 0  S(n)  S(n+1) for all n >= k • Then S(n) is true for all n >= k Sandeep Kumar Poonia
  17. 17. Proof By Induction • Claim:S(n) is true for all n >= k • Basis:  Show formula is true when n = k • Inductive hypothesis:  Assume formula is true for an arbitrary n • Step:  Show that formula is then true for n+1 Sandeep Kumar Poonia
  18. 18. Induction Example: Gaussian Closed Form • Prove 1 + 2 + 3 + … + n = n(n+1) / 2  Basis: o If n = 0, then 0 = 0(0+1) / 2  Inductive hypothesis: o Assume 1 + 2 + 3 + … + n = n(n+1) / 2  Step (show true for n+1): 1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1) = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2 Sandeep Kumar Poonia
  19. 19. Induction Example: Geometric Closed Form • Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for all a  1  Basis: show that a0 = (a0+1 - 1)/(a - 1) a0 = 1 = (a1 - 1)/(a - 1)  Inductive hypothesis: o Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1)  Step (show true for n+1): a0 + a1 + … + an+1 = a0 + a1 + … + an + an+1 = (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1) Sandeep Kumar Poonia
  20. 20. Induction • • We’ve been using weak induction Strong induction also holds  Basis: show S(0)  Hypothesis: assume S(k) holds for arbitrary k <= n  Step: Show S(n+1) follows • Another variation:  Basis: show S(0), S(1)  Hypothesis: assume S(n) and S(n+1) are true  Step: show S(n+2) follows Sandeep Kumar Poonia
  21. 21. Asymptotic Performance • Here, we care most about asymptotic performance  How does the algorithm behave as the problem size gets very large? o Running time o Memory/storage requirements o Bandwidth/power requirements/logic gates/etc. Sandeep Kumar Poonia
  22. 22. Asymptotic Notation • By now you should have an intuitive feel for asymptotic (big-O) notation:  What does O(n) running time mean? O(n2)? O(n lg n)?  How does asymptotic running time relate to asymptotic memory usage? • Our first task is to define this notation more formally and completely Sandeep Kumar Poonia
  23. 23. Analysis of Algorithms • Analysis is performed with respect to a • computational model We will usually use a generic uniprocessor random-access machine (RAM)  All memory equally expensive to access  No concurrent operations  All reasonable instructions take unit time o Except, of course, function calls  Constant word size o Unless we are explicitly manipulating bits Sandeep Kumar Poonia
  24. 24. Input Size • Time and space complexity  This is generally a function of the input size o E.g., sorting, multiplication  How we characterize input size depends: o Sorting: number of input items o Multiplication: total number of bits o Graph algorithms: number of nodes & edges Etc Sandeep Kumar Poonia
  25. 25. Running Time • Number of primitive steps that are executed  Except for time of executing a function call most statements roughly require the same amount of time oy=m*x+b o c = 5 / 9 * (t - 32 ) o z = f(x) + g(y) • We can be more exact if need be Sandeep Kumar Poonia
  26. 26. Analysis • Worst case  Provides an upper bound on running time  An absolute guarantee • Average case  Provides the expected running time  Very useful, but treat with care: what is “average”? o Random (equally likely) inputs o Real-life inputs Sandeep Kumar Poonia
  27. 27. An Example: Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  28. 28. An Example: Insertion Sort 30 10 40 20 1 2 3 4 i =  j =  key =  A[j] =  A[j+1] =  InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  29. 29. An Example: Insertion Sort 30 10 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 10 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  30. 30. An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  31. 31. An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=1 A[j] = 30 key = 10 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  32. 32. An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=0 A[j] =  key = 10 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  33. 33. An Example: Insertion Sort 30 30 40 20 1 2 3 4 i=2 j=0 A[j] =  key = 10 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  34. 34. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=2 j=0 A[j] =  key = 10 A[j+1] = 10 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  35. 35. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] =  key = 10 A[j+1] = 10 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  36. 36. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] =  key = 40 A[j+1] = 10 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  37. 37. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=0 A[j] =  key = 40 A[j+1] = 10 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  38. 38. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  39. 39. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  40. 40. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=3 j=2 A[j] = 30 key = 40 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  41. 41. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 40 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  42. 42. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  43. 43. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  44. 44. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 20 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  45. 45. An Example: Insertion Sort 10 30 40 20 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 20 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  46. 46. An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  47. 47. An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  48. 48. An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=3 A[j] = 40 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  49. 49. An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  50. 50. An Example: Insertion Sort 10 30 40 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 40 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  51. 51. An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  52. 52. An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=2 A[j] = 30 key = 20 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  53. 53. An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  54. 54. An Example: Insertion Sort 10 30 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 30 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  55. 55. An Example: Insertion Sort 10 20 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 20 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  56. 56. An Example: Insertion Sort 10 20 30 40 1 2 3 4 i=4 j=1 A[j] = 10 key = 20 A[j+1] = 20 InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Done! Sandeep Kumar Poonia
  57. 57. Insertion Sort What is the precondition InsertionSort(A, n) { for this loop? for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } } Sandeep Kumar Poonia
  58. 58. Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } How many times will } this loop execute? Sandeep Kumar Poonia
  59. 59. Insertion Sort Statement Effort InsertionSort(A, n) { for i = 2 to n { c1n key = A[i] c2(n-1) j = i - 1; c3(n-1) while (j > 0) and (A[j] > key) { c4T A[j+1] = A[j] c5(T-(n-1)) j = j - 1 c6(T-(n-1)) } 0 A[j+1] = key c7(n-1) } 0 } T = t2 + t3 + … + tn where ti is number of while expression evaluations for the ith for loop iteration Sandeep Kumar Poonia
  60. 60. Analyzing Insertion Sort • • T(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1) = c8T + c9n + c10 What can T be?  Best case -- inner loop body never executed o ti = 1  T(n) is a linear function  Worst case -- inner loop body executed for all previous elements o ti = i  T(n) is a quadratic function Sandeep Kumar Poonia
  61. 61. Analysis • Simplifications  Ignore actual and abstract statement costs  Order of growth is the interesting measure: o Highest-order term is what counts   Sandeep Kumar Poonia Remember, we are doing asymptotic analysis As the input size grows larger it is the high order term that dominates
  62. 62. Upper Bound Notation • We say InsertionSort’s run time is O(n2)  Properly we should say run time is in O(n2)  Read O as “Big-O” (you’ll also hear it as “order”) • In general a function  f(n) is O(g(n)) if there exist positive constants c and n0 such that f(n)  c  g(n) for all n  n0 • Formally  O(g(n)) = { f(n):  positive constants c and n0 such that f(n)  c  g(n)  n  n0 Sandeep Kumar Poonia
  63. 63. Lower Bound Notation • We say InsertionSort’s run time is (n) • In general a function  f(n) is (g(n)) if  positive constants c and n0 such that 0  cg(n)  f(n)  n  n0 • Proof:  Suppose run time is an + b o Assume a and b are positive (what if b is negative?)  an  an + b Sandeep Kumar Poonia
  64. 64. Asymptotic Tight Bound • A function f(n) is (g(n)) if  positive constants c1, c2, and n0 such that c1 g(n)  f(n)  c2 g(n)  n  n0 • Theorem  f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n)) Sandeep Kumar Poonia
  65. 65. Practical Complexity 250 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Sandeep Kumar Poonia
  66. 66. Practical Complexity 500 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Sandeep Kumar Poonia
  67. 67. Practical Complexity 1000 f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n 0 1 Sandeep Kumar Poonia 3 5 7 9 11 13 15 17 19
  68. 68. Practical Complexity 5000 4000 f(n) = n f(n) = log(n) 3000 f(n) = n log(n) f(n) = n^2 2000 f(n) = n^3 f(n) = 2^n 1000 0 1 Sandeep Kumar Poonia 3 5 7 9 11 13 15 17 19
  69. 69. Practical Complexity 10000000 1000000 100000 10000 1000 100 10 1 1 Sandeep Kumar Poonia 4 16 64 256 1024 4096 16384 65536
  70. 70. Other Asymptotic Notations • A function f(n) is o(g(n)) if  positive • • constants c and n0 such that f(n) < c g(n)  n  n0 A function f(n) is (g(n)) if  positive constants c and n0 such that c g(n) < f(n)  n  n0 Intuitively,  o() is like <  O() is like  Sandeep Kumar Poonia  () is like >  () is like   () is like =

×