Upcoming SlideShare
×

2007spring Algo00

622 views

Published on

Published in: Technology, Business
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

• Be the first to like this

Views
Total views
622
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
6
0
Likes
0
Embeds 0
No embeds

No notes for slide

2007spring Algo00

1. 1. 資料結構與演算法 ( 下 ) 呂學一 (Hsueh-I Lu)
2. 2. Today <ul><li>Randomized quick sort </li></ul><ul><li>Randomized maximum cut </li></ul>
3. 3. Sorting <ul><li>Input: n distinct numbers x 1 , x 2 , …, x n . </li></ul><ul><li>Output: a sorted list of x 1 , x 2 , …, x n . </li></ul>4 3 5 8 1 9 2 6 7 4 3 5 8 1 9 2 6 7
4. 4. Quick sort: divide & conquer a l g o r i t h m q u i c k s o r t ( X ) f i f X i s e m p t y t h e n r e t u r n ; s e l e c t a n u m b e r x ; l e t Y = f y 2 X j y < x g ; l e t Z = f z 2 X j z > x g ; c a l l q u i c k s o r t ( Y ) ; p r i n t x ; c a l l q u i c k s o r t ( Z ) ; g
5. 5. x divides X into Y and Z. 4 2 7 8 1 9 3 6 5 4 x 4 2 7 8 1 9 3 6 5 Y Z
6. 6. The whole process 4 2 7 8 1 9 3 6 5 2 1 3 4 7 8 9 6 5 1 2 3 6 5 7 8 9 5 9 1 3 5 6 8 9
7. 7. Efficiency depends on … Critical step a l g o r i t h m q u i c k s o r t ( X ) f i f X i s e m p t y t h e n r e t u r n ; s e l e c t a n u m b e r x ; l e t Y = f y 2 X j y < x g ; l e t Z = f z 2 X j z > x g ; c a l l q u i c k s o r t ( Y ) ; p r i n t x ; c a l l q u i c k s o r t ( Z ) ; g
8. 8. A bad case 9 8 7 6 5 4 3 2 1 9 1 2 3 4 5 6 7 8 8 1 2 3 4 5 6 7 7 1 2 3 4 5 6 6 1 2 3 4 5 5 1 2 3 4 4 1 2 3 3 1 2 2 1 1 ­ ( n )
9. 9. A good case <ul><li>Each x divides X evenly into Y and Z , i.e., when –1 · | Y | – | Z | · 1. </li></ul><ul><li>The running time is O ( n log n ). </li></ul>O ( l o g n )
10. 10. Median selection <ul><li>If the median-selection problem can be solved in O ( n ) time, then algorithm quicksort runs in O ( n log n ) time. </li></ul>F I n p u t : a ( m u l t i p l e ) s e t X o f n n u m b e r s . F O u t p u t : a n u m b e r x i n X w i t h ¡ 1 · j f y 2 X j y < x g j ¡ j f z 2 X j z > x g j · 1 .
11. 11. How hard is median selection? <ul><li>[Blum et al. STOC ’72 & JCSS ’73] </li></ul><ul><ul><li>A “shining” paper by five authors: </li></ul></ul><ul><ul><ul><li>Manuel Blum (Turing Award 1995) </li></ul></ul></ul><ul><ul><ul><li>Robert W. Floyd (Turing Award 1978) </li></ul></ul></ul><ul><ul><ul><li>Vaughan R. Pratt </li></ul></ul></ul><ul><ul><ul><li>Ronald L. Rivest (Turing Award 2002) </li></ul></ul></ul><ul><ul><ul><li>Robert E. Tarjan (Turing Award 1986) </li></ul></ul></ul><ul><ul><li>The number of comparisons required to find a median is between 1.5 n and 5.43 n . </li></ul></ul>
12. 12. Number of comparisonss <ul><li>Upper bound: </li></ul><ul><ul><li>3 n + o ( n ) by Schonhage, Paterson, and Pippenger ( JCSS 1975). </li></ul></ul><ul><ul><li>2.95 n by Dor and Zwick ( SODA 1995, SIAM Journal on Computing 1999). </li></ul></ul><ul><li>Lower bound: </li></ul><ul><ul><li>2 n + o ( n ) by Bent and John ( STOC 1985) </li></ul></ul><ul><ul><li>(2+2 -80 ) n by Dor and Zwick ( FOCS 1996, SIAM Journal on Discrete Math 2001). </li></ul></ul>
13. 13. Question: Do we really need all the complication of median selection to obtain an O ( n log n )-time quicksort? Perhaps not!
14. 14. For example, x does not have to be a median. O ( l o g n ) A s l o n g a s j Y j j Z j = £ ( 1 ) , t h e r e s u l t i n g q u i c k s o r t r u n s i n O ( n l o g n ) t i m e .
15. 15. How many good choices for x? O b s e r v a t i o n F o r e x a m p l e , i f w e o n l y a i m f o r 1 3 · j Y j j Z j · 3 ; t h e n a t l e a s t h a l f o f t h e e l e m e n t s x i n X a r e g o o d .
16. 16. This leads to RandQS 隨機快排法
17. 17. Randomized quicksort randomization a l g o r i t h m R a n d Q S ( X ) f i f X i s e m p t y t h e n r e t u r n ; s e l e c t x u n i f o r m l y a t r a n d o m f r o m X ; l e t Y = f y 2 X j y < x g ; l e t Z = f z 2 X j z > x g ; c a l l R a n d Q S ( Y ) ; p r i n t x ; c a l l R a n d Q S ( Z ) ; g
18. 18. 2 Questions for RandQS <ul><li>Is RandQS correct? </li></ul><ul><ul><li>That is, does RandQS “always ” output a sorted list of X ? </li></ul></ul><ul><li>What is the time complexity of RandQS? </li></ul><ul><ul><li>Due to the randomization for selecting x , the running time for RandQS becomes a random variable . </li></ul></ul><ul><ul><li>We are interested in the expected time complexity for RandQS. </li></ul></ul>
19. 19. Features for many randomized algorithms <ul><li>Implementation is relatively simple, </li></ul><ul><ul><li>compared to their deterministic counterparts. </li></ul></ul><ul><li>The challenge for designing randomized algorithms are usually in (asymptotic) analysis for (time and space) complexity. </li></ul>
20. 20. Let C = #comparision. <ul><li>When we choose x uniformly at random from X , the probability for each element of X being chosen as x is exactly 1/| X |. </li></ul><ul><li>The values of C for two independent executions of RandQS are very likely to be different. </li></ul><ul><li>So, C is a random variable . We are interested in determining E [ C ] , since the expected running time is clearly O ( E [ C ]) . </li></ul>
21. 21. C o m p u t i n g E [ C ] f r o m E [ C i ; j ] S u p p o s e X c o n s i s t s o f x 1 ; : : : ; x n w i t h x 1 < ¢ ¢ ¢ < x n . L e t C i ; j d e n o t e t h e n u m b e r o f t i m e s t h a t x i a n d x j a r e c o m p a r e d d u r i n g t h e e x e c u t i o n o f R a n d Q S . B y t h e l i n e a r i t y o f E [ ¢ ] , w e h a v e E [ C ] = E 2 4 n X i = 1 X j > i C i ; j 3 5 = n X i = 1 X j > i E [ C i ; j ] :
22. 22. 4 2 7 8 1 9 3 6 5 2 1 3 4 7 8 9 6 5 1 2 3 6 5 7 8 9 5 9 1 3 5 6 8 9 C i ; j c a n o n l y b e 0 o r 1 .
23. 23. Therefore, E [ C i ; j ] = P r [ C i ; j = 1 ] ¢ 1 + P r [ C i ; j = 0 ] ¢ 0 = P r [ C i ; j = 1 ] : Q : W h a t i s t h e v a l u e o f P r [ C i ; j = 1 ] ? D o y o u t h i n k P r [ C i ; j = 1 ] h a s a ¯ x e d v a l u e i n d i ® e r - e n t e x e c u t i o n s o f R a n d Q S ( X ) ?
24. 24. F o r e x a m p l e , P r [ C 1 ; n = 1 ] = 2 n ¡ 1 + 1 = 2 n : R e a s o n : x 1 a n d x n a r e c o m p a r e d ( o n c e ) i f a n d o n l y i f t h e ¯ r s t c h o i c e o f x i s e i t h e r x 1 o r x n . C L A I M P r [ C i ; j = 1 ] = 2 j ¡ i + 1 :
25. 25. Another example F o r e x a m p l e , P r [ C 2 ; n = 1 ] ? = 2 n ¡ 2 + 1 = 2 n ¡ 1 : Y e s : x 1 a n d x n a r e c o m p a r e d ( o n c e ) i f a n d o n l y i f ( a ) t h e ¯ r s t c h o i c e o f x i s e i t h e r x 2 o r x n , o r ( b ) t h e ¯ r s t c h o i c e o f x i s x 1 a n d t h e s e c o n d c h o i c e o f x i s e i t h e r x 2 o r x n . T h e r e f o r e , P r [ C 2 ; n = 1 ] = 2 n + 1 n £ 2 n ¡ 1 = 2 n ¡ 1 : W o w !
26. 26. It seems hard to further generalize the argument … We need a more general approach to prove the amazingly simple claim.
27. 27. The subset X i,j of X L e t X i ; j b e t h e s u b s e t o f X s u c h t h a t R a n d Q S ( X i ; j ) i s t h e ¯ r s t s u b r o u t i n e c a l l s u c h t h a t i t s x b e l o n g s t o t h e s e t f x i ; x i + 1 ; : : : ; x j g . C l e a r l y , f x i ; : : : ; x j g µ X i ; j . T h i s s p e c i ¯ c c h o i c e o f x d e t e r m i n e s w h e t h e r x i a n d x j w i l l e v e r b e c o m p a r e d t o e a c h o t h e r t h r o u g h o u t t h e e x e c u t i o n I n p a r t i c u l a r , C i ; j = 1 i f a n d o n l y i f t h i s c h o i c e o f x a m o n g f x i ; x i + 1 ; : : : ; x j g i s e i t h e r x i o r x j . T h e r e f o r e , P r [ C i ; j = 1 ] = 2 j ¡ i + 1 :
28. 28. For example, 2 7 4 8 1 9 3 6 5 3 1 2 4 8 9 7 6 5 1 2 3 5 6 7 8 9 5 9 1 3 5 6 8 9 X 3,6 = X 2,4 = X 1,9 X 8,9 X 1,3 = X 1,2 = X 2,3
29. 29. Comments about X i,j <ul><li>For each execution of RandQS( X ), X i,j is well defined. </li></ul><ul><li>However, the X i,j for two independent execution of RandQS( X ) are (very likely to be) different. </li></ul>
30. 30. So, E[C] = O(n log n) E [ C ] = n X i = 1 X j > i E [ C i ; j ] = n X i = 1 X j > i 2 j ¡ i + 1 = n X i = 1 n ¡ i X k = 1 2 k + 1 · 2 n µ 1 + 1 2 + 1 3 + ¢ ¢ ¢ + 1 n ¶ = O ( n l o g n )
31. 31. In summary, T h e e x p e c t e d r u n n i n g t i m e o f R a n d Q S i s O ( E [ C ] ) = O ( n l o g n ) . a l g o r i t h m R a n d Q S ( X ) f i f X i s e m p t y t h e n r e t u r n ; s e l e c t x u n i f o r m l y a t r a n d o m f r o m X ; l e t Y = f y 2 X j y · x g ; l e t Z = f z 2 X j z > x g ; c a l l R a n d Q S ( Y ) ; p r i n t x ; c a l l R a n d Q S ( Z ) ; g
32. 32. Features for many randomized algorithms <ul><li>Implementation is relatively simple, </li></ul><ul><ul><li>compared to their deterministic counterparts. </li></ul></ul><ul><li>The challenge for designing randomized algorithms are usually in (asymptotic) analysis for (time and space) complexity. </li></ul>
33. 33. A misunderstanding <ul><li>“ The performance of randomized algorithms depends on the distribution of the input.” </li></ul><ul><li>Clarification: Not at all (for this course) </li></ul><ul><ul><li>For example, our analysis for RandQS did not assume that n ! possible permutations for the input are given to RandQS with probability 1/( n !). More precisely, the expected running time for RandQS is O ( n log n ) on ANY input. </li></ul></ul>
34. 34. <ul><li>Probabilistic analysis for a ( deterministic or randomized ) algorithm assumes a particular probability distribution on the input and then studies the expected performance of the algorithm. </li></ul>RA  probabilistic analysis
35. 35. Achilles’ heel of RA <ul><li>Is there a random number generator? (That is, can computer throw a die?) </li></ul>
36. 36. Related to CMI’s Millennium Problems <ul><li>Pseudo-random generator exists if and only if one-way function exists. </li></ul><ul><li>If one-way function exists, then P  NP . </li></ul><ul><li>Anybody proving or disproving P  NP gets US\$1,000,000 from Clay Math Institute, which raised 7 open questions in year 2000, imitating Hilbert’s 23 open questions raised in year 1900. </li></ul>
37. 37. CMI Millennium Prize <ul><li>Clay Mathematics Institute (Cambridge, MA, USA) offered US\$100,000 for each of seven open problems on May 24, 2000 at Paris. </li></ul><ul><ul><li>| Birch and Swinnerton -Dyer Conjecture | Hodge Conjecture | Navier -Stokes Equations | P vs NP | Poincare Conjecture | Riemann Hypothesis | Yang-Mills Theory | </li></ul></ul>
38. 38. Simulate a fair coin by a biased coin. <ul><li>By von Neumann (1951): </li></ul><ul><ul><li>biased “head” + biased “tail”  fair “head”. </li></ul></ul><ul><ul><li>biased “tail” + biased “head”  fair “tail”. </li></ul></ul><ul><ul><li>Otherwise, repeat the above procedure. </li></ul></ul>
39. 39. Part 2 掃黑的藝術 (maximum cut)
40. 40. Maximum Cut Problem <ul><li>Input: </li></ul><ul><ul><li>A graph G </li></ul></ul><ul><li>Output: </li></ul><ul><ul><li>A partition of G ’s nodes into A and B that maximizes the number of edges between A and B . </li></ul></ul>
41. 41. Illustration
42. 42. Intractability <ul><li>The problem is NP-hard even if each node in the input graph has no more than 3 neighbors. </li></ul>Bad News
43. 43. How “hard” is NP-hard? <ul><li>So far, scientist only knows how to solve such a problem on an n -node graph in O ( c n ) time for some constant c . </li></ul>
44. 44. That is, … <ul><li>“ Impossible” to find an optimal solution for all graphs with, say, 10000 nodes. </li></ul>
45. 45. On the bright side… <ul><li>If you can come up with an algorithm for such a problem that runs in O ( n 1000000 ) time, then you will be awarded Turing Award for sure plus US\$1,000,000 from CMI. </li></ul>
46. 46. NP-completeness = Dead end ?
47. 47. 退一步 海闊天空 Approximation Algorithms ( 近似演算法 )
48. 48. 近似演算法的哲學 放下對完美的堅持 , 往往就可以找到新的出路。 知所進退 , 則近道矣 !
49. 49. A extremely simple randomized approximation For each node v of G , put v into A with probability ½.
50. 50. Approximation Algorithm <ul><li>Criterion 1: feasibility </li></ul><ul><ul><li>output a feasible solution. </li></ul></ul><ul><li>Criterion 2: tractability </li></ul><ul><ul><li>runs in polynomial time. </li></ul></ul><ul><li>Criterion 3: quality </li></ul><ul><ul><li>The solution’s quality is provably not too far from that of an optimal solution. </li></ul></ul>
51. 51. Q1: feasibility <ul><li>Any partition is a feasible solution. </li></ul>
52. 52. Q2: tractability <ul><li>Runs in (deterministic) linear time, with the help of a random number generator. </li></ul>
53. 53. Q3: quality? <ul><li>The expected approximation ratio is 2. </li></ul><ul><li>More precisely, we can prove that the expected cardinality of the output cut is at least 0.5 times that of an optimal cut. </li></ul><ul><ul><li>Note that the proof for the expected approximation ratio does not require knowing the cardinality of an optimal in advance. </li></ul></ul>
54. 54. Let C = the cardinality of the output cut. L e t m b e t h e n u m b e r o f e d g e s i n t h e i n p u t g r a p h . C l e a r l y , t h e s i z e o f a n y c u t , i n c l u d i n g a n y m a x i m u m c u t , i s a t m o s t m . W e h a v e E [ C ] = X e d g e e o f G P r [ e i s i n t h e o u t p u t c u t ] = m 2 : S o , t h e e x p e c t e d a p p r o x i m a t i o n r a t i o i s a t m o s t 2 .
55. 55. Interesting enough, this simple algorithm was the best previously known approximation until… M. Goemans and D. Williamson, [ACM STOC 1994]
56. 56. A seminal paper <ul><li>1/0.878-randomized approximation for MAXCUT. </li></ul><ul><li>Initiate a series of research in Operations Research, Scientific Computing, and Approximation Algorithms. </li></ul>
57. 57. Traditional approach Integer Linear Program Integral solution Linear Program Fractional solution relaxation Rounding Approximation Solver
58. 58. SDP-based approach Quadratic Program Scalar solution Semi-definite Program Vector solution relaxation Rounding Approximation Solver
59. 59. Remarks <ul><li>勇於嘗試 , 努力創新 </li></ul><ul><li>一篇好作品的影響 , 往往遠甚於許多普通的論文 . ( 跳高 ) </li></ul>