- 1. Growth of Functions Lecture 8, CMSC 56 Allyn Joy D. Calcaben
- 2. Big – O Notation
- 3. Used to measure how well a computer algorithm scales as the amount of data involved increases. Big – O Notation
- 4. Used to measure how well a computer algorithm scales as the amount of data involved increases. Not a measure of speed but a measure of how well an algorithm scales. Big – O Notation
- 5. Big – O Notation is also known as Landau Notation Named after Edmund Landau He didn’t invent the notation, he popularized it Ironic that he popularized Big – O as he was known to be an exact and meticulous mathematician Landau Notation
- 6. If n = 1: 45n3 + 20n2 + 19 = Example
- 7. If n = 1: 45n3 + 20n2 + 19 = 84 Example
- 8. If n = 1: 45n3 + 20n2 + 19 = 84 If n = 2: 45n3 + 20n2 + 19 = Example
- 9. If n = 1: 45n3 + 20n2 + 19 = 84 If n = 2: 45n3 + 20n2 + 19 = 459 Example
- 10. If n = 1: 45n3 + 20n2 + 19 = 84 If n = 2: 45n3 + 20n2 + 19 = 459 If n = 10: 45n3 + 20n2 + 19 = Example
- 11. If n = 1: 45n3 + 20n2 + 19 = 84 If n = 2: 45n3 + 20n2 + 19 = 459 If n = 10: 45n3 + 20n2 + 19 = 47, 019 45,000 + 2,000 + 19 Example
- 12. If n = 100: 45n3 + 20n2 + 19 Removing Unimportant Things
- 13. If n = 100: 45n3 + 20n2 + 19 = 45, 000, 000 + 200, 000 + 19 Removing Unimportant Things
- 14. If n = 100: 45n3 + 20n2 + 19 = 45, 000, 000 + 200, 000 + 19 = 45, 200, 019 Removing Unimportant Things
- 15. If n = 100: 45n3 + 20n2 + 19 = 45, 000, 000 + 200, 000 + 19 = 45, 200, 019 = O(N3) Removing Unimportant Things
- 16. “Leaving away all the unnecessary things that we don’t care about when comparing algorithms.” Big – O Notation
- 17. 2n2 + 23 vs. 2n2 + 27 Example
- 18. 2n2 + 23 vs. 2n2 + 27 Essentially they have the same running time. This is another simplification. Solution
- 19. Definition 1 Given A = f(n), B = g(n) grows faster than f(n), then there exists a number n’ and a constant c so that c•g(n’) ≥ f(n’) for all n’’ > n, we have g(n’’) ≥ f(n’’). Big – O Notation
- 20. Definition 1 Given A = f(n), B = g(n) grows faster than f(n), then there exists a number n’ and a constant c so that c•g(n’) ≥ f(n’) for all n’’ > n, we have g(n’’) ≥ f(n’’). Since we don’t want to care about constants, we include c so that we can scale g(n). That is, even if we multiply g(n) with a constant c, and it still outgrows f(n), we can still say that g(n) runs at least as fast as f(n). Big – O Notation
- 21. If the two conditions hold true, we say that f(n) ϵ O(g(n)) f(n) is contained in Big O of g(n) Big O means that g(n) is a function that runs at least as fast as f(n). Big – O Notation
- 22. f(n) g(n)h(n) n Comparing the Growth Rates
- 23. Get ¼ and answer. Which of the ff. is true? Write True or False for each number. 1. h(n) ϵ O(f(n)) 2. h(n) ϵ O(g(n)) 3. g(n) ϵ O(f(n)) 4. g(n) ϵ O(h(n)) 5. f(n) ϵ O(g(n)) 6. f(n) ϵ O(h(n)) Comparing the Growth Rates f(n) g(n)h(n) n
- 24. Get ¼ and answer. Which of the ff. is true? Write True or False for each number. 1. h(n) ϵ O(f(n)) 2. h(n) ϵ O(g(n)) 3. g(n) ϵ O(f(n)) 4. g(n) ϵ O(h(n)) 5. f(n) ϵ O(g(n)) 6. f(n) ϵ O(h(n)) Comparing the Growth Rates f(n) g(n)h(n) n
- 25. Allow us to concentrate on the fastest growing part of the function and leave out the involved constants Importance
- 26. Simply about looking at which part of the function grows the fastest. A convenient way of describing the growth of the function while ignoring the distracting and unnecessary details Big – O Notation
- 27. f(n) = O(g(n)) is also an accepted notation f(n) ϵ O(g(n)) is more accurate, since O(g(n)) is a set of functions. Big – O Notation
- 28. Algorithm A RT: 3n2 – n + 10 Algorithm B RT: 2n – 50n + 256 Which algorithm is preferable in general? Exercise
- 29. Algorithm A RT: 3n2 – n + 10 Algorithm B RT: 2n – 50n + 256 Which algorithm is preferable in general? But, this is for general cases. For small inputs, algorithm B might be equal to algorithm A or better. Solution
- 30. Algorithm A RT: 3n2 – n + 10 Algorithm B RT: 2n – 50n + 256 If n = 5: 3n2 – n + 10 = 80 2n – 50n + 256 = 38 If n = 100: 3n2 – n + 10 = 29910 2n – 50n + 256 = 1267650600228229401496703200632 Analysis
- 31. 1. 3n + 1 ϵ _______ 2. 18n2 – 50 ϵ _______ 3. 9n4 + 18 ϵ _______ 4. 30n6 + 2n + 123 ϵ _______ 5. 2nn2 + 30n6 + 123 ϵ _______ More Example
- 32. 1. 3n + 1 ϵ O(n) 2. 18n2 – 50 ϵ O(n2) 3. 9n4 + 18 ϵ O(n4) 4. 30n6 + 2n + 123 ϵ O(2n) 5. 2nn2 + 30n6 + 123 ϵ O(2n n2) More Example
- 33. O(1) O(log N) O(N) O(N log N) O(N2) O(N3) O(2n) Big – O Notation
- 34. 3n + 1 ϵ O(n2) True / False? More Examples
- 35. 3n + 1 ϵ O(n2) True / False? TRUE, but NOT TIGHT More Examples
- 36. 18n2 – 50 ϵ O(n3) True / False? If true, tightly bound / not? If not tightly bound, what is the tighter bound? More Examples
- 37. 18n2 – 50 ϵ O(n3) True / False? TRUE If true, tightly bound / not? NOT TIGHTLY BOUND If not tightly bound, what is the tighter bound? O(n2) More Examples
- 38. Write Correct / Incorrect, Tight / Not Tight 1. 4n2 - 300n + 12 ∈ O(n2) 2. 4n2 - 300n + 12 ∈ O(n3) 3. 3n + 5n2 - 3n ∈ O(n2) 4. 3n + 5n2 - 3n ∈ O(4n) 5. 3n + 5n2 - 3n ∈ O(3n) 6. 50•2nn2 + 5n - log(n) ∈ O(2n) On Your Seats (1/4)
- 39. Write Correct / Incorrect, Tight / Not Tight 1. 4n2 - 300n + 12 ∈ O(n2) CORRECT, TIGHT 2. 4n2 - 300n + 12 ∈ O(n3) CORRECT, NOT TIGHT 3. 3n + 5n2 - 3n ∈ O(n2) INCORRECT, NOT TIGHT 4. 3n + 5n2 - 3n ∈ O(4n) CORRECT, NOT TIGHT 5. 3n + 5n2 - 3n ∈ O(3n) CORRECT, TIGHT 6. 50•2nn2 + 5n - log(n) ∈ O(2n) INCORRECT, NOT TIGHT On Your Seats (1/4)
- 40. O(1) O(log N) O(N) O(N log N) O(N2) O(N3) O(2n) Big – O Notation
- 41. O(1)
- 42. Constant Description: Statement Example: Adding two numbers c = a + b O(1)
- 43. Algorithm that will execute at the same amount of time regardless of the amount of data. Or simply, Code that will execute at the same amount of time no matter how big the array is. O(1)
- 44. Example: Adding element to array list.append(x) O(1)
- 45. O(log N)
- 46. Logarithmic Description: Divide in Half Data is decreased roughly 50% each time through the algorithm O(log N)
- 47. Example: Binary Search O(log N)
- 48. O(N)
- 49. Linear Description: Loop Time to complete will grow in direct proportion to the amount of data. Example: Factorial of N using Loops for i in range (1, N+1): factorial *= i O(N)
- 50. Example: Finding the Maximum O(N)
- 51. Example: Finding the Maximum Look in exactly each item in the array Big difference if it was a 10 – item array vs. a 10 thousand – item array O(N)
- 53. O(n log N)
- 54. Linearithmic / Loglinear Description: Divide and Conquer O(n log N)
- 55. Example: Merge Sort O(n log N)
- 56. Example: Merge Sort O(n log N)
- 57. O(N2)
- 58. Quadratic Description: Double Loop Time to complete will grow proportional to the square of amount of data O(N2)
- 61. O(N3)
- 63. O(2n)
- 64. Exponential Description: Exhaustive Search (Brute Force*) O(2n)
- 65. Implications
- 66. Given an algorithm that runs in ___ time, the computer can solve a problem size of _____ in a matter of minutes. Practical Implications
- 67. Given an algorithm that runs in ___ time, the computer can solve a problem size of _____ in a matter of minutes. Constant: any Logarithmic: any Linear: billions Loglinear: hundreds of millions Quadratic: tens of thousands Cubic: thousands Exponential: 30 Practical Implications
- 68. Best Case, Worst Case
- 69. count = 0 for each character in string: if character == ‘a’: count += 1 Best Case, Worst Case
- 70. Best case: 2n + 1 Worst case: 3n Running Time
- 71. We observe the ff. from the algorithm: 1. Each character in the string is considered at most once 2. For each character in the input string, a constant no. of steps is performed (2 or 3) Running Time
- 72. We observe the ff. from the algorithm: 1. Each character in the string is considered at most once 2. For each character in the input string, a constant no. of steps is performed (2 or 3) With Big – O, we say c1n + c2 ϵ O(n) We say that the running time of the algorithm is O(n) or linear time Running Time
- 73. Any Questions?

- Ex. How well will it work using a 10 – element array vs. a 10 thousand – element array?
- Ex. How well will it work using a 10 – element array vs. a 10 thousand – element array?
- Ex. How well will it work using a 10 – element array vs. a 10 thousand – element array?
- Ex. How well will it work using a 10 – element array vs. a 10 thousand – element array?
- Algorithm A
- True. n2 grows at least as fast as 3n + 1. But this is unusual because we usually try to make the bound as tight as possible.
- Constant: any Logarithmic: any Linear: billions Loglinear: hundreds of millions Quadratic: tens of thousands Cubic: thousands Exponential: 30