Measuring Cost Asymptotic Operators

1. Class 18:
Measuring
Cost
cs1120 Fall 2011
David Evans
Colossus Rebuilt, Bletchley Park, Summer 2004 3 October 2011

2. Plan
How Computer Scientists Measure Cost
Asymptotic Operators
Assistant Coaches’ Review Sessions for Exam 1:
Tuesday (tomorrow), 6:30pm, Rice 442
Wednesday, 7:30pm, Rice 442
2

3. (define (fibo-rec n) (define (fibo-loop n)
(if (= n 0) 0 (cdr
(if (= n 1) 1 (loop 1 (cons 0 1) (lambda (i) (< i n)) inc
(+ (fibo-rec (- n 1)) (lambda (i v)
(fibo-rec (- n 2)))))) (cons (cdr v) (+ (car v) (cdr v)))))))
> (time (fibo-rec 2)) > (time (fibo-loop 2))
cpu time: 0 real time: 0 gc time: 0 cpu time: 0 real time: 0 gc time: 0
1 1
> (time (fibo-rec 5)) > (time (fibo-loop 5))
cpu time: 0 real time: 0 gc time: 0 cpu time: 0 real time: 0 gc time: 0
5 5
> (time (fibo-rec 20)) > (time (fibo-loop 20))
cpu time: 0 real time: 2 gc time: 0 cpu time: 0 real time: 0 gc time: 0
6765 6765
> (time (fibo-rec 30)) > (time (fibo-loop 30))
cpu time: 359 real time: 370 gc time: 0 cpu time: 0 real time: 0 gc time: 0
832040 832040
> (time (fibo-rec 60)) > (time (fibo-loop 60))
still waiting since Friday… cpu time: 0 real time: 0 gc time: 0
1548008755920

4. (define (fibo-rec n) (define (fibo-loop n)
(if (= n 0) 0 (cdr
(if (= n 1) 1 (loop 1 (cons 0 1)
(+ (fibo-rec (- n 1)) (lambda (i) (< i n)) inc
(fibo-rec (- n 2)))))) (lambda (i v)
(cons (cdr v) (+ (car v) (cdr v)))))))
Number of “expensive” calls: Number of “expensive” calls:
where n is the value of the input,
and is the “golden ratio”:

5. How Computer Scientists
Measure Cost
Abstract: hide all the details that will change
when you get your next laptop to capture the
fundamental cost of the procedure
Cost: number of steps for a Turing Machine to
execute the procedure
Order of Growth: what matters is how the cost
scales with the size of the input
Size of input: how many TM squares needed to
represent it
Usually, we can determine these without actually writing a TM version of our procedure!
5

6. Orders of Growth
6

7. Asymptotic Operators
7

8. Asymptotic Operators
These notations define sets of functions
In computing, the function inside the operator is
(usually) a mapping from the size of the input to the
number of steps required. We use the asymptotic
operators to abstract away all the silly details about
particular computers.
8

9. Big O
Intuition: the set of functions that grow no
faster than f (more formal definition soon)
Asymptotic growth rate: as input to f approaches
infinity, how fast does value of f increase
Hence, only the fastest-growing term in f matters.
?
?
9

10. Examples
O(n3) f(n) = 12n2 + n
O(n2)
f(n) = n2.5
f(n) = n3.1 – n2
10

11. Formal Definition
11

12. O Examples
x O (x2)?
10x O (x)?
x2 O (x)?
12

13. Lower Bound: (Omega)
Only difference from O: this was
13

14. Where is
(n2)?
O(n3) f(n) = 12n2 + n
O(n2)
f(n) = n2.5 (n2)
f(n) = n3.1 – n2
14

15. Inside-Out
(n3) f(n) = 12n2 + n
(n2)
f(n) = n2.5 O(n2)
f(n) = n3.1 – n2
15

16. Recap
Big-O: functions that grow no faster than f
Omega ( ): functions that grow no slower than f
16

17. The Sets O(f ) and Ω(f )
O(f) Ω(f) Functions
Functions that grow
that grow no slower
no faster f than f
than f
17

18. What else might be useful?
O(n3) f(n) = 12n2 + n
O(n2)
f(n) = n2.5 (n2)
f(n) = n3.1 – n2
18

19. Theta (“Order of”)
Intuition: set of functions that grow as fast as f
Definition:
Slang: When people say, “f is order g” that means
19

20. Tight Bound Theta ( )
O(n3) f(n) = 12n2 + n
O(n2)
f(n) = n2.5 (n2)
(n2)
f(n) = n3.1 – n2 Faster Growing
20

21. Θ Examples
Is 10n in Θ(n)?
Yes, since 10n is (n) and 10n is in O(n)
Doesn’t matter that you choose different c values
for each part; they are independent
Is n2 in Θ(n)?
No, since n2 is not in O(n)
Is n in Θ(n2)?
No, since n2 is not in (n)
21

22. The Sets O(f ), Ω(f ), and Θ(f )
O(f) Ω(f) Functions
Functions that grow
that grow no slower
no faster f than f
than f
Θ(f)
How big are O(f ), Ω(f ), and Θ(f )?
22

23. Summary
Big-O: grow no faster than f
Omega: grow no slower than f
Theta:
Which of these would we most like to know about
costproc(n) = number of steps to execute proc on
input of size n
?
23

24. Complexity of Problems
So, why do we need O and Ω?
Computer scientists often care about the
complexity of problems not algorithms. The
complexity of a problem is the complexity of the
best possible algorithm that solves the problem.
24

25. Algorithm Analysis
What is the asymptotic running time of the
Scheme procedure below:
(define (bigger a b)
(if (> a b) a b))
25

26. Charge
Next class:
analyzing the costs of bigger
analyzing the costs of brute-force-lorenz
Assistant Coaches’ Review Sessions for Exam 1:
Tuesday (tomorrow), 6:30pm, Rice 442
Wednesday, 7:30pm, Rice 442
26