Upcoming SlideShare
×

# Joel Spencer – Finding Needles in Exponential Haystacks

235 views

Published on

We discuss two recent methods in which an object with a certain property is sought. In both, using of a straightforward random object would succeed with only exponentially small probability. The new randomized algorithms run efficiently and also give new proofs of the existence of the desired object. In both cases there is a potentially broad use of the methodology.

(i) Consider an instance of k-SAT in which each clause overlaps (has a variable in common, regardless of the negation symbol) with at most d others. Lovasz showed that when ed < 2k (regardless of the number of variables) the conjunction of the clauses was satisfiable. The new approach due to Moser is to start with a random true-false assignment. In a WHILE loop, if any clause is not satisfied we ”fix it” by a random reassignment. The analysis of the algorithm is unusual, connecting the running of the algorithm with certain Tetris patterns, and leading to some algebraic combinatorics. [These results apply in a quite general setting with underlying independent ”coin flips” and bad events (the clause not being satisfied) that depend on only a few of the coin flips.]

(ii) No Outliers. Given n vectors rj in n-space with all coefficients in [−1,+1] one wants a vector x = (x1, ..., xn) with all xi = +1 or −1 so that all dot products x · rj are at most K √ n in absolute value, K an absolute constant. A random x would make x · rj Gaussian but there would be outliers. The existence of such an x was first shown by the speaker. The first algorithm was found by Nikhil Bansal. The approach here, due to Lovett and Meka, is to begin with x = (0, ..., 0) and let it float in a kind of restricted Brownian Motion until all the coordinates hit the boundary.

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
235
On SlideShare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
5
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Joel Spencer – Finding Needles in Exponential Haystacks

1. 1. Finding Needles in Exponential Haystacks Joel Spencer June 6, 2014 Yandex - Moscow
2. 2. Erd˝os Magic: If a random object has a positive probability of being good than a good object MUST exist
3. 3. Modern Erd˝os Magic: If a randomized algorithm has a positive probability of producing a good object than a good object MUST exist
4. 4. Working with Paul Erd˝os was like taking a walk in the hills. Every time when I thought that we had achieved our goal and deserved a rest, Paul pointed to the top of another hill and oﬀ we would go. – Fan Chung
5. 5. PART I The Lov´asz Local Lemma
6. 6. k-SAT Boolean x1, . . . , xn. ys is xs or xs Clause C = yi1 ∨ . . . ∨ yik k-SAT instance: ∧α∈I Cα Cα, Cβ overlap if common xj . Assume: Each Cα overlaps ≤ d Cβ Assume: ed2−k ≤ 1 LLL: Then satisﬁable.
7. 7. k-SAT Fix d, k (e.g.: k = 5, d = 10) but let n → ∞ Where is the satisfying assignment? Original Proof: Can’t Find it
8. 8. k-SAT Fix d, k (e.g.: k = 5, d = 10) but let n → ∞ Where is the satisfying assignment? Original Proof: Can’t Find it MOSER: I can ﬁnd it!
9. 9. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f }
10. 10. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f
11. 11. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα
12. 12. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα
13. 13. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα FIX-IT IV
14. 14. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα FIX-IT IV There is no IV!
15. 15. The LOG LOG - Clauses reassigned in order FIX-IT IIIb applies TLOG = length of LOG. (= ∞ if no stop) Modern Erd˝os Magic: E[TLOG] < ∞ implies satisﬁable and a good 1 algorithm Example: Variables 12345678. Clauses A : 123, B : 234, C : 345, D : 456, E : 567, F : 678. LOG = ADCFECBF 1 with E[TLOG] reasonable
16. 16. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A A A - - - - - 1 2 3 4 5 6 7 8
17. 17. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A A A D D D - - 1 2 3 4 5 6 7 8
18. 18. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - C C C - - - A A A D D D - - 1 2 3 4 5 6 7 8
19. 19. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
20. 20. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
21. 21. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - C C C - - - - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
22. 22. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - B B B - - - - - - C C C - - - - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
23. 23. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - B B B - - - - - - C C C F F F - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
24. 24. Let’s Play Tetris! s = ADCFECBF - B B B - - - - - - C C C F F F - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8 The Pyramid Pyr(s) = ADCFEF is support of last: - - - - - F F F - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
26. 26. Preprocess Randomness Each xj chooses countably many assignments. Probability X1 · · · Xt is pyramid of preﬁx is ≤ (2−k )t - - C C C - - - A A A - - - - - 1 2 3 4 5 6 7 8 A, C false with diﬀerent coinﬂips!
27. 27. An Interesting Algebra X, Y commute if no overlap. Tetris same if and only if equal in algebra ADC = DAC - - C C C - - - A A A D D D - - 1 2 3 4 5 6 7 8 ADC, DAC use same coinﬂips.
28. 28. Algebraic Combinatorics Property KNUTH: s(2−k )length(s) < ∞ (sum over pyramids s in algebra) E[TLOG] ≤ s(2−k )length(s) Modern Erd˝os Magic: [KNUTH] implies satisﬁability and FIX-IT takes “time” at most sum. Algebraic Combinatorics: When does [KNUTH] hold? Partial Answer: if ed2−k ≤ 1
29. 29. A Prescient Adversary FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα Each xj selects countably many t, f . Adversary knows coinﬂips in advance Still can’t stop FIX-IT from halting!
30. 30. PART II Eliminating Outliers
31. 31. Six Standard Deviations Suﬃce S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n
32. 32. Six Standard Deviations Suﬃce S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n Conjecture (JS/1986-2009) You can’t ﬁnd χ in polynomial time.
33. 33. Six Standard Deviations Suﬃce S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n Conjecture (JS/1986-2009) You can’t ﬁnd χ in polynomial time. Theorem (Bansal/2010): Yes I can!
34. 34. Six Standard Deviations Suﬃce S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n Conjecture (JS/1986-2009) You can’t ﬁnd χ in polynomial time. Theorem (Bansal/2010): Yes I can! Theorem (Lovett, Meka/2012): We can too!
35. 35. A Vector Formulation ri ∈ Rn, 1 ≤ i ≤ n, |ri |∞ ≤ 1 Initial z ∈ [−1, +1]n (e.g.: z = 0.) Theorem: There exists x ∈ {−1, +1}n with |ri · (x − z)| ≤ K √ n for all 1 ≤ i ≤ n.
36. 36. A Vector Formulation ri ∈ Rn, 1 ≤ i ≤ n, |ri |∞ ≤ 1 Initial z ∈ [−1, +1]n (e.g.: z = 0.) Theorem: There exists x ∈ {−1, +1}n with |ri · (x − z)| ≤ K √ n for all 1 ≤ i ≤ n. z = 0, x random. Problem: OUTLIERS!
37. 37. Phase I Find x ∈ [−1, +1]n with all least n 2 at ±1. Idea: Start x ← z. Move x in a Controlled Brownian Motion.
38. 38. Phase I Find x ∈ [−1, +1]n with all least n 2 at ±1. Idea: Start x ← z. Move x in a Controlled Brownian Motion. Technical: i frozen if |xi | ≥ 1 − ǫ. Each step of distance δ
39. 39. Phase I Find x ∈ [−1, +1]n with all least n 2 at ±1. Idea: Start x ← z. Move x in a Controlled Brownian Motion. Technical: i frozen if |xi | ≥ 1 − ǫ. Each step of distance δ Set Lj = [n−1/2rj ] · [x − z] WANT: ALL |Lj | ≤ K
40. 40. Space V of allowable moves y = (y1, . . . , yn)
41. 41. Space V of allowable moves y = (y1, . . . , yn) i frozen ⇒ yi = 0.
42. 42. Space V of allowable moves y = (y1, . . . , yn) i frozen ⇒ yi = 0. y orthogonal to current x
43. 43. Space V of allowable moves y = (y1, . . . , yn) i frozen ⇒ yi = 0. y orthogonal to current x KEY: y orthogonal to rj for j with top n 4 |Lj |
44. 44. The Random Move d = dim(V ) ≥ n 4 − 1 ∼ n 4 . Gaussian g = d−1/2[n1b1 + . . . + nd bd ], orthonormal bs Move x ← x + δg
45. 45. The Random Move d = dim(V ) ≥ n 4 − 1 ∼ n 4 . Gaussian g = d−1/2[n1b1 + . . . + nd bd ], orthonormal bs Move x ← x + δg Lj ← Lj + δ[n−1/2rj ] · g Gaussian Nondirectional Lj ← Lj + N(0, τ2) with τ2 ≤ 1 d δ2 ≤ 4 n δ2
46. 46. Gaussian Casino Martingale 0 = X0, X1, . . . , XT . ∆t = Xt − Xt−1 ∼ N(0, τ2) with τ2 ≤ σ2 t σ2 := σ2 1 + . . . + σ2 T Theorem: Pr[XT ≥ λσ] ≤ exp[−λ2/2] Theorem: Pr[max0≤t≤T Xt ≥ λσ] ≤ exp[−λ2/2]
47. 47. |x|2 ← |x|2 + δ2. Set T = 4nδ−2 (ﬁctitious continuation). Fix j. Xt value of Lj after step t. Martingale! σ2 = T · 4 n δ2 = 16 Pr[|Lj | ≥ K] ≤ 2e−K2/32 ≤ 0.1 Pr[|Lj | ≥ K at any time ] ≤ 0.1. SUCCESS: Fewer than n 5 j with |Lj | ≥ K. Positive Probability of SUCCESS
48. 48. |x|2 ← |x|2 + δ2. Set T = 4nδ−2 (ﬁctitious continuation). Fix j. Xt value of Lj after step t. Martingale! σ2 = T · 4 n δ2 = 16 Pr[|Lj | ≥ K] ≤ 2e−K2/32 ≤ 0.1 Pr[|Lj | ≥ K at any time ] ≤ 0.1. SUCCESS: Fewer than n 5 j with |Lj | ≥ K. Positive Probability of SUCCESS SUCCESS implies that ALL |Lj | ≤ K
49. 49. Phase s m = 21−s n. Start z with ≤ m coordinates frozen. End x with ≤ m 2 coordinates frozen. Eﬀectively |rj | ≤ √ m Would get K √ m but still have n = m2s−1 vectors. Actually get: K √ m √ s = K √ n √ s2(1−s)/2 Converges!
50. 50. Thank You! It is six in the morning. The house is asleep. Nice music is playing. I prove and conjecture. – Paul Erd˝os, in letter to Vera S´os