Finding Needles in Exponential Haystacks
Joel Spencer
June 6, 2014
Yandex - Moscow
Erd˝os Magic:
If a random object has a positive probability of being
good than a good object MUST exist
Modern Erd˝os Magic:
If a randomized algorithm has a positive probability of
producing a good object than a good object MU...
Working with Paul Erd˝os was like taking a walk in the hills. Every
time when I thought that we had achieved our goal and ...
PART I
The Lov´asz Local Lemma
k-SAT
Boolean x1, . . . , xn. ys is xs or xs
Clause C = yi1 ∨ . . . ∨ yik
k-SAT instance: ∧α∈I Cα
Cα, Cβ overlap if common...
k-SAT
Fix d, k (e.g.: k = 5, d = 10) but let n → ∞
Where is the satisfying assignment?
Original Proof: Can’t Find it
k-SAT
Fix d, k (e.g.: k = 5, d = 10) but let n → ∞
Where is the satisfying assignment?
Original Proof: Can’t Find it
MOSER...
FIX-IT!
Moser’s FIX-IT Algorithm
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT!
Moser’s FIX-IT Algorithm
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT II WHILE some clause Cα ← f
FIX-IT!
Moser’s FIX-IT Algorithm
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT II WHILE some clause Cα ← f
FIX-IT IIIa Sele...
FIX-IT!
Moser’s FIX-IT Algorithm
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT II WHILE some clause Cα ← f
FIX-IT IIIa Sele...
FIX-IT!
Moser’s FIX-IT Algorithm
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT II WHILE some clause Cα ← f
FIX-IT IIIa Sele...
FIX-IT!
Moser’s FIX-IT Algorithm
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT II WHILE some clause Cα ← f
FIX-IT IIIa Sele...
The LOG
LOG - Clauses reassigned in order FIX-IT IIIb applies
TLOG = length of LOG. (= ∞ if no stop)
Modern Erd˝os Magic: ...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- - - - - - - -
- - - - - - - -
- - - - - - - -
- - - - - - - -
A A A - - - - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- - - - - - - -
- - - - - - - -
- - - - - - - -
- - - - - - - -
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- - - - - - - -
- - - - - - - -
- - - - - - - -
- - C C C - - -
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- - - - - - - -
- - - - - - - -
- - - - - - - -
- - C C C F F F
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- - - - - - - -
- - - - - - - -
- - - - E E E -
- - C C C F F F
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- - - - - - - -
- - C C C - - -
- - - - E E E -
- - C C C F F F
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- B B B - - - -
- - C C C - - -
- - - - E E E -
- - C C C F F F
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF: ADCFECBF
- B B B - - - -
- - C C C F F F
- - - - E E E -
- - C C C F F F
A A A D D D - -
...
Let’s Play Tetris!
s = ADCFECBF
- B B B - - - -
- - C C C F F F
- - - - E E E -
- - C C C F F F
A A A D D D - -
1 2 3 4 5 ...
Analysis
Pyramids of prefixes of LOG distinct.
ADCFECBF: A; D; ADC;
DF;ADCFE;ADCFEC;ADCFECB;ADCFEF
E[TLOG] = s Pr[s pyramid...
Preprocess Randomness
Each xj chooses countably many assignments.
Probability X1 · · · Xt is pyramid of prefix is ≤ (2−k )t...
An Interesting Algebra
X, Y commute if no overlap.
Tetris same if and only if equal in algebra
ADC = DAC
- - C C C - - -
A...
Algebraic Combinatorics
Property KNUTH: s(2−k )length(s) < ∞
(sum over pyramids s in algebra)
E[TLOG] ≤ s(2−k )length(s)
M...
A Prescient Adversary
FIX-IT I Randomly assign xj ← {t, f }
FIX-IT II WHILE some clause Cα ← f
FIX-IT IIIa Select one bad ...
PART II
Eliminating Outliers
Six Standard Deviations Suffice
S1, . . . , Sn ⊆ {1, . . . , n}
χ : {1, . . . , n} → {−1 + 1} = {red, blue}
χ(S) := j∈S χ(j)...
Six Standard Deviations Suffice
S1, . . . , Sn ⊆ {1, . . . , n}
χ : {1, . . . , n} → {−1 + 1} = {red, blue}
χ(S) := j∈S χ(j)...
Six Standard Deviations Suffice
S1, . . . , Sn ⊆ {1, . . . , n}
χ : {1, . . . , n} → {−1 + 1} = {red, blue}
χ(S) := j∈S χ(j)...
Six Standard Deviations Suffice
S1, . . . , Sn ⊆ {1, . . . , n}
χ : {1, . . . , n} → {−1 + 1} = {red, blue}
χ(S) := j∈S χ(j)...
A Vector Formulation
ri ∈ Rn, 1 ≤ i ≤ n, |ri |∞ ≤ 1
Initial z ∈ [−1, +1]n (e.g.: z = 0.)
Theorem: There exists x ∈ {−1, +1...
A Vector Formulation
ri ∈ Rn, 1 ≤ i ≤ n, |ri |∞ ≤ 1
Initial z ∈ [−1, +1]n (e.g.: z = 0.)
Theorem: There exists x ∈ {−1, +1...
Phase I
Find x ∈ [−1, +1]n with all least n
2 at ±1.
Idea: Start x ← z. Move x in a Controlled Brownian Motion.
Phase I
Find x ∈ [−1, +1]n with all least n
2 at ±1.
Idea: Start x ← z. Move x in a Controlled Brownian Motion.
Technical:...
Phase I
Find x ∈ [−1, +1]n with all least n
2 at ±1.
Idea: Start x ← z. Move x in a Controlled Brownian Motion.
Technical:...
Space V of allowable moves y = (y1, . . . , yn)
Space V of allowable moves y = (y1, . . . , yn)
i frozen ⇒ yi = 0.
Space V of allowable moves y = (y1, . . . , yn)
i frozen ⇒ yi = 0.
y orthogonal to current x
Space V of allowable moves y = (y1, . . . , yn)
i frozen ⇒ yi = 0.
y orthogonal to current x
KEY: y orthogonal to rj for j...
The Random Move
d = dim(V ) ≥ n
4 − 1 ∼ n
4 .
Gaussian g = d−1/2[n1b1 + . . . + nd bd ], orthonormal bs
Move x ← x + δg
The Random Move
d = dim(V ) ≥ n
4 − 1 ∼ n
4 .
Gaussian g = d−1/2[n1b1 + . . . + nd bd ], orthonormal bs
Move x ← x + δg
Lj...
Gaussian Casino
Martingale 0 = X0, X1, . . . , XT .
∆t = Xt − Xt−1 ∼ N(0, τ2) with τ2 ≤ σ2
t
σ2 := σ2
1 + . . . + σ2
T
The...
|x|2 ← |x|2 + δ2. Set T = 4nδ−2 (fictitious continuation).
Fix j. Xt value of Lj after step t.
Martingale!
σ2 = T · 4
n δ2 ...
|x|2 ← |x|2 + δ2. Set T = 4nδ−2 (fictitious continuation).
Fix j. Xt value of Lj after step t.
Martingale!
σ2 = T · 4
n δ2 ...
Phase s
m = 21−s n. Start z with ≤ m coordinates frozen. End x with ≤ m
2
coordinates frozen.
Effectively |rj | ≤
√
m
Would...
Thank You!
It is six in the morning.
The house is asleep.
Nice music is playing.
I prove and conjecture.
– Paul Erd˝os, in...
Upcoming SlideShare
Loading in …5
×

Joel Spencer – Finding Needles in Exponential Haystacks

235 views

Published on

We discuss two recent methods in which an object with a certain property is sought. In both, using of a straightforward random object would succeed with only exponentially small probability. The new randomized algorithms run efficiently and also give new proofs of the existence of the desired object. In both cases there is a potentially broad use of the methodology.

(i) Consider an instance of k-SAT in which each clause overlaps (has a variable in common, regardless of the negation symbol) with at most d others. Lovasz showed that when ed < 2k (regardless of the number of variables) the conjunction of the clauses was satisfiable. The new approach due to Moser is to start with a random true-false assignment. In a WHILE loop, if any clause is not satisfied we ”fix it” by a random reassignment. The analysis of the algorithm is unusual, connecting the running of the algorithm with certain Tetris patterns, and leading to some algebraic combinatorics. [These results apply in a quite general setting with underlying independent ”coin flips” and bad events (the clause not being satisfied) that depend on only a few of the coin flips.]

(ii) No Outliers. Given n vectors rj in n-space with all coefficients in [−1,+1] one wants a vector x = (x1, ..., xn) with all xi = +1 or −1 so that all dot products x · rj are at most K √ n in absolute value, K an absolute constant. A random x would make x · rj Gaussian but there would be outliers. The existence of such an x was first shown by the speaker. The first algorithm was found by Nikhil Bansal. The approach here, due to Lovett and Meka, is to begin with x = (0, ..., 0) and let it float in a kind of restricted Brownian Motion until all the coordinates hit the boundary.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
235
On SlideShare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Joel Spencer – Finding Needles in Exponential Haystacks

  1. 1. Finding Needles in Exponential Haystacks Joel Spencer June 6, 2014 Yandex - Moscow
  2. 2. Erd˝os Magic: If a random object has a positive probability of being good than a good object MUST exist
  3. 3. Modern Erd˝os Magic: If a randomized algorithm has a positive probability of producing a good object than a good object MUST exist
  4. 4. Working with Paul Erd˝os was like taking a walk in the hills. Every time when I thought that we had achieved our goal and deserved a rest, Paul pointed to the top of another hill and off we would go. – Fan Chung
  5. 5. PART I The Lov´asz Local Lemma
  6. 6. k-SAT Boolean x1, . . . , xn. ys is xs or xs Clause C = yi1 ∨ . . . ∨ yik k-SAT instance: ∧α∈I Cα Cα, Cβ overlap if common xj . Assume: Each Cα overlaps ≤ d Cβ Assume: ed2−k ≤ 1 LLL: Then satisfiable.
  7. 7. k-SAT Fix d, k (e.g.: k = 5, d = 10) but let n → ∞ Where is the satisfying assignment? Original Proof: Can’t Find it
  8. 8. k-SAT Fix d, k (e.g.: k = 5, d = 10) but let n → ∞ Where is the satisfying assignment? Original Proof: Can’t Find it MOSER: I can find it!
  9. 9. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f }
  10. 10. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f
  11. 11. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα
  12. 12. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα
  13. 13. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα FIX-IT IV
  14. 14. FIX-IT! Moser’s FIX-IT Algorithm FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα FIX-IT IV There is no IV!
  15. 15. The LOG LOG - Clauses reassigned in order FIX-IT IIIb applies TLOG = length of LOG. (= ∞ if no stop) Modern Erd˝os Magic: E[TLOG] < ∞ implies satisfiable and a good 1 algorithm Example: Variables 12345678. Clauses A : 123, B : 234, C : 345, D : 456, E : 567, F : 678. LOG = ADCFECBF 1 with E[TLOG] reasonable
  16. 16. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A A A - - - - - 1 2 3 4 5 6 7 8
  17. 17. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - A A A D D D - - 1 2 3 4 5 6 7 8
  18. 18. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - C C C - - - A A A D D D - - 1 2 3 4 5 6 7 8
  19. 19. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - - - - - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
  20. 20. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - - - - - - - - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
  21. 21. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - - - - - - - - - - C C C - - - - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
  22. 22. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - B B B - - - - - - C C C - - - - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
  23. 23. Let’s Play Tetris! s = ADCFECBF: ADCFECBF - B B B - - - - - - C C C F F F - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
  24. 24. Let’s Play Tetris! s = ADCFECBF - B B B - - - - - - C C C F F F - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8 The Pyramid Pyr(s) = ADCFEF is support of last: - - - - - F F F - - - - E E E - - - C C C F F F A A A D D D - - 1 2 3 4 5 6 7 8
  25. 25. Analysis Pyramids of prefixes of LOG distinct. ADCFECBF: A; D; ADC; DF;ADCFE;ADCFEC;ADCFECB;ADCFEF E[TLOG] = s Pr[s pyramid of prefix ]
  26. 26. Preprocess Randomness Each xj chooses countably many assignments. Probability X1 · · · Xt is pyramid of prefix is ≤ (2−k )t - - C C C - - - A A A - - - - - 1 2 3 4 5 6 7 8 A, C false with different coinflips!
  27. 27. An Interesting Algebra X, Y commute if no overlap. Tetris same if and only if equal in algebra ADC = DAC - - C C C - - - A A A D D D - - 1 2 3 4 5 6 7 8 ADC, DAC use same coinflips.
  28. 28. Algebraic Combinatorics Property KNUTH: s(2−k )length(s) < ∞ (sum over pyramids s in algebra) E[TLOG] ≤ s(2−k )length(s) Modern Erd˝os Magic: [KNUTH] implies satisfiability and FIX-IT takes “time” at most sum. Algebraic Combinatorics: When does [KNUTH] hold? Partial Answer: if ed2−k ≤ 1
  29. 29. A Prescient Adversary FIX-IT I Randomly assign xj ← {t, f } FIX-IT II WHILE some clause Cα ← f FIX-IT IIIa Select one bad clause Cα FIX-IT IIIb Randomly Reassign xj ← {t, f } for all xj in Cα Each xj selects countably many t, f . Adversary knows coinflips in advance Still can’t stop FIX-IT from halting!
  30. 30. PART II Eliminating Outliers
  31. 31. Six Standard Deviations Suffice S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n
  32. 32. Six Standard Deviations Suffice S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n Conjecture (JS/1986-2009) You can’t find χ in polynomial time.
  33. 33. Six Standard Deviations Suffice S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n Conjecture (JS/1986-2009) You can’t find χ in polynomial time. Theorem (Bansal/2010): Yes I can!
  34. 34. Six Standard Deviations Suffice S1, . . . , Sn ⊆ {1, . . . , n} χ : {1, . . . , n} → {−1 + 1} = {red, blue} χ(S) := j∈S χ(j), disc(S) = |χ(S)| = |red − blue| Theorem (JS/1985): There exists χ disc(Si) ≤ 6 √ n for all 1 ≤ i ≤ n Conjecture (JS/1986-2009) You can’t find χ in polynomial time. Theorem (Bansal/2010): Yes I can! Theorem (Lovett, Meka/2012): We can too!
  35. 35. A Vector Formulation ri ∈ Rn, 1 ≤ i ≤ n, |ri |∞ ≤ 1 Initial z ∈ [−1, +1]n (e.g.: z = 0.) Theorem: There exists x ∈ {−1, +1}n with |ri · (x − z)| ≤ K √ n for all 1 ≤ i ≤ n.
  36. 36. A Vector Formulation ri ∈ Rn, 1 ≤ i ≤ n, |ri |∞ ≤ 1 Initial z ∈ [−1, +1]n (e.g.: z = 0.) Theorem: There exists x ∈ {−1, +1}n with |ri · (x − z)| ≤ K √ n for all 1 ≤ i ≤ n. z = 0, x random. Problem: OUTLIERS!
  37. 37. Phase I Find x ∈ [−1, +1]n with all least n 2 at ±1. Idea: Start x ← z. Move x in a Controlled Brownian Motion.
  38. 38. Phase I Find x ∈ [−1, +1]n with all least n 2 at ±1. Idea: Start x ← z. Move x in a Controlled Brownian Motion. Technical: i frozen if |xi | ≥ 1 − ǫ. Each step of distance δ
  39. 39. Phase I Find x ∈ [−1, +1]n with all least n 2 at ±1. Idea: Start x ← z. Move x in a Controlled Brownian Motion. Technical: i frozen if |xi | ≥ 1 − ǫ. Each step of distance δ Set Lj = [n−1/2rj ] · [x − z] WANT: ALL |Lj | ≤ K
  40. 40. Space V of allowable moves y = (y1, . . . , yn)
  41. 41. Space V of allowable moves y = (y1, . . . , yn) i frozen ⇒ yi = 0.
  42. 42. Space V of allowable moves y = (y1, . . . , yn) i frozen ⇒ yi = 0. y orthogonal to current x
  43. 43. Space V of allowable moves y = (y1, . . . , yn) i frozen ⇒ yi = 0. y orthogonal to current x KEY: y orthogonal to rj for j with top n 4 |Lj |
  44. 44. The Random Move d = dim(V ) ≥ n 4 − 1 ∼ n 4 . Gaussian g = d−1/2[n1b1 + . . . + nd bd ], orthonormal bs Move x ← x + δg
  45. 45. The Random Move d = dim(V ) ≥ n 4 − 1 ∼ n 4 . Gaussian g = d−1/2[n1b1 + . . . + nd bd ], orthonormal bs Move x ← x + δg Lj ← Lj + δ[n−1/2rj ] · g Gaussian Nondirectional Lj ← Lj + N(0, τ2) with τ2 ≤ 1 d δ2 ≤ 4 n δ2
  46. 46. Gaussian Casino Martingale 0 = X0, X1, . . . , XT . ∆t = Xt − Xt−1 ∼ N(0, τ2) with τ2 ≤ σ2 t σ2 := σ2 1 + . . . + σ2 T Theorem: Pr[XT ≥ λσ] ≤ exp[−λ2/2] Theorem: Pr[max0≤t≤T Xt ≥ λσ] ≤ exp[−λ2/2]
  47. 47. |x|2 ← |x|2 + δ2. Set T = 4nδ−2 (fictitious continuation). Fix j. Xt value of Lj after step t. Martingale! σ2 = T · 4 n δ2 = 16 Pr[|Lj | ≥ K] ≤ 2e−K2/32 ≤ 0.1 Pr[|Lj | ≥ K at any time ] ≤ 0.1. SUCCESS: Fewer than n 5 j with |Lj | ≥ K. Positive Probability of SUCCESS
  48. 48. |x|2 ← |x|2 + δ2. Set T = 4nδ−2 (fictitious continuation). Fix j. Xt value of Lj after step t. Martingale! σ2 = T · 4 n δ2 = 16 Pr[|Lj | ≥ K] ≤ 2e−K2/32 ≤ 0.1 Pr[|Lj | ≥ K at any time ] ≤ 0.1. SUCCESS: Fewer than n 5 j with |Lj | ≥ K. Positive Probability of SUCCESS SUCCESS implies that ALL |Lj | ≤ K
  49. 49. Phase s m = 21−s n. Start z with ≤ m coordinates frozen. End x with ≤ m 2 coordinates frozen. Effectively |rj | ≤ √ m Would get K √ m but still have n = m2s−1 vectors. Actually get: K √ m √ s = K √ n √ s2(1−s)/2 Converges!
  50. 50. Thank You! It is six in the morning. The house is asleep. Nice music is playing. I prove and conjecture. – Paul Erd˝os, in letter to Vera S´os

×