Complexity         www.tudorgirba.com
computationinformation                 information              computer
memory                                 bandwidthComputation requires resources                                 time       ...
1 public long factorial (int n) {2 	 long factorial = 1;3 	 int i = 1;4 	 while ( i ≤ n ) {5 	 	 factorial = factorial * i...
1 public long factorial (int n) {2 	 long factorial = 1;             13 	 int i = 1;                      14 	 while ( i ≤...
1 public long factorial (int n) {2 	 long factorial = 1;             1   13 	 int i = 1;                      1   14 	 whi...
1 public long factorial (int n) {2 	 long factorial = 1;                 1       13 	 int i = 1;                          ...
Big O Notationf: N -> Ng: N -> Nf ∈ O(g)	 ∃ c, n0 :	 (c,n0∈N) ∧ (c>0) :	 (∀n : n∈N ∧ n>n0 : f(n) ≤ c*g(n))
Big O Notationf: N -> Ng: N -> Nf ∈ O(g)	 ∃ c, n0 :	 (c,n0∈N) ∧ (c>0) :	 (∀n : n∈N ∧ n>n0 : f(n) ≤ c*g(n))f: N % N, f(n)=1...
!"#$%$&()("*$+ ,-.$/--O(1)      ConstantO(log n) LogarithmicO(n)      LinearO(n2)     QuadraticO(nk)     PolynomialO(kn)  ...
O(1) ∈ O(log n) ∈ O(n) ∈ O(n2) ∈ O(nk) ∈ O(2n)                        !"#$%$&()("*$+ ,-.$/--O(1)      ConstantO(log n) Log...
N        log N N log N      N2      2N      N!   10         3       33       100     1024   2*106   20         4       86 ...
N        log N N log N      N2       2N          N!   10         3       33       100     1024       2*106   20         4 ...
O(c*f) = O(f)O(f) + O(g) = O(f+g)            = max{O(f),O(g)}O(f*g) = O(f)*O(g)
O(c*f) = O(f)O(f) + O(g) = O(f+g)            = max{O(f),O(g)}O(f*g) = O(f)*O(g)O(f) ⊆ O(g)     f ∈ O(g)O(f) = O(g)     (O(...
a b c d e f g                                    a 0 2 3 0 0 0 0a                       e           b 0 0 1 0 0 0 0    3  ...
a b c d e f g                                        a 0 2 3 0 0 0 0 a                       e              b 0 0 1 0 0 0 ...
O(2*n-1)                       = O(n)O(n*(n+1)/2)                   = O(n2)O(log n2)                      = O(log n)O((3*n...
O(2*n-1)                       = O(n)O(n*(n+1)/2)                   = O(n2)O(log n2)                      = O(log n)O((3*n...
4   2   5   1    6            3                                         ch                           : Line ar sear       ...
4   2   5   1    6            3                                         ch                           : Line ar sear       ...
4   2   5   1    6            3                                         ch                           : Line ar sear       ...
Worst case: 	 	 n stepsBest case:	 	 1 stepAverage case:	 n/2 steps  4        2       5       1    6            3         ...
Best case estimationf: N -> Ng: N -> Nf ∈ Ω(g)	 ∃ c, n0 :	 (c,n0∈N) ∧ (c>0) :	 (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))
Best case estimationf: N -> Ng: N -> Nf ∈ Ω(g)	 ∃ c, n0 :	 (c,n0∈N) ∧ (c>0) :	 (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))(3n2+6n+9)...
Best case estimationf: N -> Ng: N -> Nf ∈ Ω(g)	 ∃ c, n0 :	 (c,n0∈N) ∧ (c>0) :	 (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))(3n2+6n+9)...
Average case estimationf: N -> Ng: N -> Nf ∈ θ(g)	 ∃ c 1, c 2, n 0 :	 (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0):	 (∀n : n∈N ∧ n≥n0 : ...
Average case estimationf: N -> Ng: N -> Nf ∈ θ(g)	 ∃ c 1, c 2, n 0 :	 (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0):	 (∀n : n∈N ∧ n≥n0 : ...
Average case estimationf: N -> Ng: N -> Nf ∈ θ(g)	 ∃ c 1, c 2, n 0 :	 (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0):	 (∀n : n∈N ∧ n≥n0 : ...
Average case estimationf: N -> Ng: N -> Nf ∈ θ(g)	 ∃ c 1, c 2, n 0 :	 (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0):	 (∀n : n∈N ∧ n≥n0 : ...
boolean f ( int[][] a , int n ) {	 for ( int i = 0 ; i < n ; i++ ) {	 	 for ( int j = i + 1 ; j < n ; j++ ) {	 	 	 if ( a[...
boolean f ( int[][] a , int n ) {	 for ( int i = 0 ; i < n ; i++ ) {	 	 for ( int j = i + 1 ; j < n ; j++ ) {	 	 	 if ( a[...
4    2                          61       3             5                                  ch                    : Bina ry ...
f ∈ O(log n)f ∈ Ω(1)f ∈ θ(log n)                   4        2                             6 1             3             5 ...
solvable in O(nk)
solvable in O(nk)   P
solvable in O(nk)   P   NP
solvable in O(nk)   P   NP   verifiable                     in O(nk)
solvable in O(nk)   P ⊂ NP   verifiable                     in O(nk)
solvable in O(nk)   P = NP              ?      verifiable                     in O(nk)
NP-complete NP, and one cannot do better
Hamiltonian patha                  e     c      d               gb                  f                                     ...
:               p roblemNP-co mplete sman                le  Trave ling sa
:           p roblem     mplete ingNP-co h color   Grap
S={x1, … , xn}t ∈ N∃ {y1, … , yk} ⊆ S : Σyi=t ?                                                 :                         ...
S={x1, … , xn}t ∈ N∃ {y1, … , yk} ⊆ S : Σyi=t ?S= {8, 11, 16, 29, 37}t = 37{11, 16}{8, 29}{37}                            ...
Will this program terminate?                                g problem                     Th e haltin dable               ...
Tudor Gîrba        www.tudorgirba.comcreativecommons.org/licenses/by/3.0/
08 - Complexity
08 - Complexity
Upcoming SlideShare
Loading in …5
×

08 - Complexity

951 views

Published on

I used this set of slides for the lecture on Complexity I gave at the University of Zurich for the 1st year students following the course of Formale Grundlagen der Informatik.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
951
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

08 - Complexity

  1. 1. Complexity www.tudorgirba.com
  2. 2. computationinformation information computer
  3. 3. memory bandwidthComputation requires resources time ...
  4. 4. 1 public long factorial (int n) {2 long factorial = 1;3 int i = 1;4 while ( i ≤ n ) {5 factorial = factorial * i;6 i = i + 1;7 }8 return factorial9 }
  5. 5. 1 public long factorial (int n) {2 long factorial = 1; 13 int i = 1; 14 while ( i ≤ n ) { 45 factorial = factorial * i; 46 i = i + 1; 37 }8 return factorial 19 }
  6. 6. 1 public long factorial (int n) {2 long factorial = 1; 1 13 int i = 1; 1 14 while ( i ≤ n ) { 4 n5 factorial = factorial * i; 4 n6 i = i + 1; 3 n7 }8 return factorial 1 19 }
  7. 7. 1 public long factorial (int n) {2 long factorial = 1; 1 13 int i = 1; 1 14 while ( i ≤ n ) { 4 n5 factorial = factorial * i; 4 n6 i = i + 1; 3 n7 }8 return factorial 1 19 } f(n) = 1+1+4*n+4*n+3*n+1 = 11*n+3
  8. 8. Big O Notationf: N -> Ng: N -> Nf ∈ O(g) ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≤ c*g(n))
  9. 9. Big O Notationf: N -> Ng: N -> Nf ∈ O(g) ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≤ c*g(n))f: N % N, f(n)=11*n+3.f ∈ O(n)
  10. 10. !"#$%$&()("*$+ ,-.$/--O(1) ConstantO(log n) LogarithmicO(n) LinearO(n2) QuadraticO(nk) PolynomialO(kn) Exponential
  11. 11. O(1) ∈ O(log n) ∈ O(n) ∈ O(n2) ∈ O(nk) ∈ O(2n) !"#$%$&()("*$+ ,-.$/--O(1) ConstantO(log n) LogarithmicO(n) LinearO(n2) QuadraticO(nk) PolynomialO(kn) Exponential
  12. 12. N log N N log N N2 2N N! 10 3 33 100 1024 2*106 20 4 86 400 106 2*1018 100 7 664 10’000 1031 10157 1’000 10 10’000 106 10’000 13 130’000 108 100’000 17 106 10101’000’000 1’000 2*107 1012
  13. 13. N log N N log N N2 2N N! 10 3 33 100 1024 2*106 20 4 86 400 106 2*1018 100 7 664 10’000 1031 10157 1’000 10 10’000 106 10 2 s = 1. 10’000 13 130’000 108 10 4 s = 7 minutes 2. 10 5 s = 8 hours 1.1 day 100’000 17 106 1010 10 s = 6 s 1.6 we 10 s = 7 eks 3.8 mo 10 s = 8 nths1’000’000 1’000 2*107 1012 3.1 yea 10 s = 9 rs 3.1 dec 10 10 s a = 3.1 c des 10 11 s enturie = neve s r :)
  14. 14. O(c*f) = O(f)O(f) + O(g) = O(f+g) = max{O(f),O(g)}O(f*g) = O(f)*O(g)
  15. 15. O(c*f) = O(f)O(f) + O(g) = O(f+g) = max{O(f),O(g)}O(f*g) = O(f)*O(g)O(f) ⊆ O(g) f ∈ O(g)O(f) = O(g) (O(f) ⊆ O(g)) ∧ (O(g) ⊆ O(f))O(f) ⊂ O(g) (O(f) ⊆ O(g)) ∧ (O(g) ≠ O(f))
  16. 16. a b c d e f g a 0 2 3 0 0 0 0a e b 0 0 1 0 0 0 0 3 5 3 2 c 0 0 0 2 0 0 02 c d g d 0 0 0 0 5 4 0 1 4 3 e 0 0 0 0 0 0 3b f f 0 0 0 0 0 0 3 g 0 0 0 0 0 0 0
  17. 17. a b c d e f g a 0 2 3 0 0 0 0 a e b 0 0 1 0 0 0 0 3 5 3 2 c 0 0 0 2 0 0 02 c d g d 0 0 0 0 5 4 0 1 4 3 e 0 0 0 0 0 0 3 b f f 0 0 0 0 0 0 3 g 0 0 0 0 0 0 0procedure FloydWarshall () for k := 1 to n for i := 1 to n for j := 1 to n path[i][j] = min ( path[i][j], path[i][k]+path[k][j] );
  18. 18. O(2*n-1) = O(n)O(n*(n+1)/2) = O(n2)O(log n2) = O(log n)O((3*n2 + 6*n+9)*log(1+2*n)) = O(n2log(n))
  19. 19. O(2*n-1) = O(n)O(n*(n+1)/2) = O(n2)O(log n2) = O(log n)O((3*n2 + 6*n+9)*log(1+2*n)) = O(n2log(n))
  20. 20. 4 2 5 1 6 3 ch : Line ar sear E xample
  21. 21. 4 2 5 1 6 3 ch : Line ar sear E xample
  22. 22. 4 2 5 1 6 3 ch : Line ar sear E xample
  23. 23. Worst case: n stepsBest case: 1 stepAverage case: n/2 steps 4 2 5 1 6 3 ch : Line ar sear E xample
  24. 24. Best case estimationf: N -> Ng: N -> Nf ∈ Ω(g) ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))
  25. 25. Best case estimationf: N -> Ng: N -> Nf ∈ Ω(g) ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))(3n2+6n+9)*log(1+2n) ≤ 36 n2 * log n(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
  26. 26. Best case estimationf: N -> Ng: N -> Nf ∈ Ω(g) ∃ c, n0 : (c,n0∈N) ∧ (c>0) : (∀n : n∈N ∧ n>n0 : f(n) ≥ c*g(n))(3n2+6n+9)*log(1+2n) ≤ 36 n2 * log n(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)(3n2+6n+9)*log(1+2n) ≥ n2 * log n(3n2+6n+9)*log(1+2n) ∈ Ω(n2 * log n)
  27. 27. Average case estimationf: N -> Ng: N -> Nf ∈ θ(g) ∃ c 1, c 2, n 0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))
  28. 28. Average case estimationf: N -> Ng: N -> Nf ∈ θ(g) ∃ c 1, c 2, n 0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)
  29. 29. Average case estimationf: N -> Ng: N -> Nf ∈ θ(g) ∃ c 1, c 2, n 0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)(3n2+6n+9)*log(1+2n) ∈ Ω(n2 * log n)
  30. 30. Average case estimationf: N -> Ng: N -> Nf ∈ θ(g) ∃ c 1, c 2, n 0 : (c1,c2,n0∈N) ∧ (c1>0) ∧ (c2>0): (∀n : n∈N ∧ n≥n0 : c1*g(n) ≤ f(n) ≤ c2*g(n))(3n2+6n+9)*log(1+2n) ∈ O(n2 * log n)(3n2+6n+9)*log(1+2n) ∈ Ω(n2 * log n)(3n2+6n+9)*log(1+2n) ∈ θ(n2 * log n)
  31. 31. boolean f ( int[][] a , int n ) { for ( int i = 0 ; i < n ; i++ ) { for ( int j = i + 1 ; j < n ; j++ ) { if ( a[i][j] == 0 ) {return false;} } return true; }} E xample
  32. 32. boolean f ( int[][] a , int n ) { for ( int i = 0 ; i < n ; i++ ) { for ( int j = i + 1 ; j < n ; j++ ) { if ( a[i][j] == 0 ) {return false;} } return true; }}f ∈ O(n2)f ∈ Ω(1)f ∈ θ(n2) E xample
  33. 33. 4 2 61 3 5 ch : Bina ry sear le Examp
  34. 34. f ∈ O(log n)f ∈ Ω(1)f ∈ θ(log n) 4 2 6 1 3 5 ch : Bina ry sear le Examp
  35. 35. solvable in O(nk)
  36. 36. solvable in O(nk) P
  37. 37. solvable in O(nk) P NP
  38. 38. solvable in O(nk) P NP verifiable in O(nk)
  39. 39. solvable in O(nk) P ⊂ NP verifiable in O(nk)
  40. 40. solvable in O(nk) P = NP ? verifiable in O(nk)
  41. 41. NP-complete NP, and one cannot do better
  42. 42. Hamiltonian patha e c d gb f : p roblem mplete path NP-co onian Hamilt
  43. 43. : p roblemNP-co mplete sman le Trave ling sa
  44. 44. : p roblem mplete ingNP-co h color Grap
  45. 45. S={x1, … , xn}t ∈ N∃ {y1, … , yk} ⊆ S : Σyi=t ? : p roblem NP-co mplete m u S ubset s
  46. 46. S={x1, … , xn}t ∈ N∃ {y1, … , yk} ⊆ S : Σyi=t ?S= {8, 11, 16, 29, 37}t = 37{11, 16}{8, 29}{37} : p roblem NP-co mplete m u S ubset s
  47. 47. Will this program terminate? g problem Th e haltin dable ci is unde
  48. 48. Tudor Gîrba www.tudorgirba.comcreativecommons.org/licenses/by/3.0/

×