Limits of local algorithms for random graphsLimits of local algorithms for random graphs
David Gamarnik
Joint work with
Ma...
Local Algorithms
Computer Science.
Applications: hardware, fault-tolerant models of computation,
sensor networks.
Lynch (book) [1996]
Nguye...
Network Science. Economic team modeling
Applications: models of social interaction, structure of social networks
Rusmevich...
Mathematics/Combinatorics.
Applications: graph limits.
Lovasz (book) [2012]
Hatami, Lovasz & Szegedy [2012]
Elek & Lippner...
Random n-node d-regular graph
I. Local algorithms for random graphs : i.i.d.
factors
locally regular tree like
Local algorithms: i.i.d. factors
Hatami-Lovasz-Szegedy [2012] framework
Generate i.i.d. U[0,1] weights
Apply some local ru...
Conjecture. Hatami, Lovasz & Szegedy [2012] There exists a local rule f
which produces a nearly largest independent set in...
Algorithms for independent sets in random graphs
Frieze & Luczak [1992]
Best known algorithm: Greedy
Hatami-Lovasz-Szegedy...
Theorem. [G & Sudan ] Hatami-Lovasz-Szegedy conjecture is not valid: No
local rule f can produce an independent set larger...
Geometry of solutions: clustering phenomena
Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’
independe...
Geometry of solutions: clustering phenomena
Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’
independe...
Proof sketch: argue by contradiction. Suppose rule f exists
?
(a) Generate two i.i.d. sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Proof sketch: argue by contradiction. Suppose rule f exists
(b) Interpolate between the two sequences
Intersection size be...
Theory is great not when it is correct but when
it is
interesting… Murray Davis, 1 97 1
II. Sequential local algorithms for random NAE-
K-SAT problem
Not-All-Equal-K-Satisfiability formula on n boolean variable...
Sequential local algorithms for random NAE-K-SAT
problem
Random NAE-K-SAT problem
Generate m clauses independently uniform...
Sequential local algorithms for the random NAE-K-SAT
problem
Sequential r-local algorithms
1.Fix a “local rule” which maps...
Belief Propagation and Survey Propagation algorithms – message passing type
algorithms proposed by statistical physicists....
Theorem. G & Sudan [2013] Every sequential local algorithm fails to find a
satisfying assignment when
simple algorithm
(Un...
Proof Technique
(a) Clustering property for random NAE-K-SAT
(b) Interpolation between m randomly generated solutions
(c) ...
Proof Technique
(a) Clustering property for random NAE-K-SAT
Theorem. Let
With probability approaching unity as n increase...
Some further thoughts
• The approach should apply to other problems with “symmetry”, for example
coloring of graphs. But a...
Mick can’t get satisfaction above
(from Sequential Local Algorithms)
Chvatal & Reed [1998]
’’Mick gets some (the odds are ...
Thank you
Upcoming SlideShare
Loading in...5
×

Limits of Local Algorithms for Randomly Generated Constraint Satisfaction Problems

141
-1

Published on

A major challenge in the field of random graphs is constructing fast algorithms for solving a variety of combinatorial optimization problems, such as finding largest independent set of a graph or finding a satisfying assignment in random instances of K-SAT problem. Most of the algorithms that have been successfully analyzed in the past are so-called local algorithms which rely on making decisions based on local information.

In this talk we will discuss fundamental barrier on the power of local algorithms to solve such problems, despite the conjectures put forward in the past. In particular, we refute a conjecture regarding the power of local algorithms to find nearly largest independent sets in random regular graphs. Similarly, we show that a broad class of local algorithms, including the so-called Belief Propagation and Survey Propagation algorithms, cannot find satisfying assignments in random NAE-K-SAT problem above a certain asymptotic threshold, below which even simple algorithms succeed with high probability. Our negative results exploit fascinating geometry of feasible solutions of random constraint satisfaction problems, which was first predicted by physicists heuristically and now confirmed by rigorous methods. According to this picture, the solution space exhibits a clustering property whereby the feasible solutions tend to cluster according to the underlying Hamming distance. We show that success of local algorithms would imply violation of such a clustering property thus leading to a contradiction.

Joint work with Madhu Sudan (Microsoft Research).

Published in: Science, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
141
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Limits of Local Algorithms for Randomly Generated Constraint Satisfaction Problems

  1. 1. Limits of local algorithms for random graphsLimits of local algorithms for random graphs David Gamarnik Joint work with Madhu Sudan (Microsoft Research, New England) Yandex workshop on extremal combinatorics June 24, 2014
  2. 2. Local Algorithms
  3. 3. Computer Science. Applications: hardware, fault-tolerant models of computation, sensor networks. Lynch (book) [1996] Nguyen & Onak [2008] Rubinfeld, Tamir, Vardi, Xie [2011] Suomela (survey) [2011] Wireless communications Applications: communication protocols with low overhead Shah [2008] Shin & Shah [2012] Who is interested in local algorithms?
  4. 4. Network Science. Economic team modeling Applications: models of social interaction, structure of social networks Rusmevichientong, Van Roy [2003] Judd, Kearns & Vorobeychik [2010] Borgs, Brautbar, Chayes, Khanna & Lucier [2012] Signal Processing. Machine learning. Applications: image processing, coding theory, wireless communication, gossip algorithms. Wainwright & Jordan [2008], Mezard & Montanari [2009], Shah [2008] Shin & Shah [2012] Physics. Applications: Spin glass theory Mezard & Montanari [2009] Who is interested in local algorithms?
  5. 5. Mathematics/Combinatorics. Applications: graph limits. Lovasz (book) [2012] Hatami, Lovasz & Szegedy [2012] Elek & Lippner [2010] Lyons & Nazarov [2011] Czoka & Lippner [2012] Aldous [2012]. Who is interested in local algorithms?
  6. 6. Random n-node d-regular graph I. Local algorithms for random graphs : i.i.d. factors locally regular tree like
  7. 7. Local algorithms: i.i.d. factors Hatami-Lovasz-Szegedy [2012] framework Generate i.i.d. U[0,1] weights Apply some local rule f: decorated tree ! {0,1} for every node f 0 or 1
  8. 8. Conjecture. Hatami, Lovasz & Szegedy [2012] There exists a local rule f which produces a nearly largest independent set in a random d-regular graph Example. f =1 iff the weight of the node is larger than the weights of all of its neighbors IN OUT far from optimal ! Local algorithms: i.i.d. factors
  9. 9. Algorithms for independent sets in random graphs Frieze & Luczak [1992] Best known algorithm: Greedy Hatami-Lovasz-Szegedy Conjecture: there exists f such that
  10. 10. Theorem. [G & Sudan ] Hatami-Lovasz-Szegedy conjecture is not valid: No local rule f can produce an independent set larger than factor of the optimal, for large enough d. Theorem. [Rahman & Virag 2014] No local rule f can produce an independent set larger than factor of the optimal, for large enough d. Notes: 1. Rahman & Virag’s result is the best possible: factor ½ can be achieved by local algorithms, Lauer & Wormald [2007], G & Goldberg [2010] 2. Proof technique. Spin Glass theory: geometry of large independent sets Main result
  11. 11. Geometry of solutions: clustering phenomena Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’ independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:
  12. 12. Geometry of solutions: clustering phenomena Coja-Oghlan & Efthymiou [2010], G & Sudan [2012] Every two ‘’large’’ independent sets either have a very ‘’small’’ or a very ‘’large’’ intersection:
  13. 13. Proof sketch: argue by contradiction. Suppose rule f exists ? (a) Generate two i.i.d. sequences
  14. 14. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  15. 15. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  16. 16. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  17. 17. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  18. 18. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  19. 19. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  20. 20. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences
  21. 21. Proof sketch: argue by contradiction. Suppose rule f exists (b) Interpolate between the two sequences Intersection size belongs to the ‘’non-existent interval’’
  22. 22. Theory is great not when it is correct but when it is interesting… Murray Davis, 1 97 1
  23. 23. II. Sequential local algorithms for random NAE- K-SAT problem Not-All-Equal-K-Satisfiability formula on n boolean variables and m clauses. Each clause contains K (i.e. K=3) Assignment is satisfiable if every clause is satisfied by at least one variable, and is not satisfied by at least some other variable Formula is satisfiable if there exists at least one satisfying assignment. NP-Complete.
  24. 24. Sequential local algorithms for random NAE-K-SAT problem Random NAE-K-SAT problem Generate m clauses independently uniformly at random from the space of all possible clauses (with replacement). d=m/n – clause density. Theorem. [Coja-Oglan and Panagiotou 2014] The threshold for probability that the formula is satisfiable Best algorithm Unit Clause succeeds only when Achlioptas et al. 2001 K gap!
  25. 25. Sequential local algorithms for the random NAE-K-SAT problem Sequential r-local algorithms 1.Fix a “local rule” which maps formula © with variable x to probability value. 2.For i=1,2,…,n apply rule ¿ to variables xi and depth r sub-formulas ©i rooted at xi : fix xi to be 1 with probability ¿(©i, xi), and 0 with probability 1- ¿(©i, xi) Notes: Belief Propagation (BP) guided decimation and Survey Propagation (SP) guided decimation algorithms are sequential local algorithms when the number of iterations is bounded by a constant independent of the number of variables Unit Clause algorithm is a sequential local algorithm
  26. 26. Belief Propagation and Survey Propagation algorithms – message passing type algorithms proposed by statistical physicists. Work remarkably well for low values of K (K=3,4,5). Mezard, Parisi & Zecchina [2003] Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova [2007] Mezard & Montanari (book) [2009]
  27. 27. Theorem. G & Sudan [2013] Every sequential local algorithm fails to find a satisfying assignment when simple algorithm (Unit Clause) exist no sequential local algorithms exist no solutions exist Sequential local algorithms cannot bridge the 1/K gap
  28. 28. Proof Technique (a) Clustering property for random NAE-K-SAT (b) Interpolation between m randomly generated solutions (c) Decisions for variables are localized UNSAT
  29. 29. Proof Technique (a) Clustering property for random NAE-K-SAT Theorem. Let With probability approaching unity as n increases, there does not exist m satisfying assignments such that all pairwise Hamming distances are Proof: first moment method.
  30. 30. Some further thoughts • The approach should apply to other problems with “symmetry”, for example coloring of graphs. But applying the approach to random K-SAT is problematic. • In practice Survey Propagation algorithm is run for many iterations. Challenge: establish similar result when there is no bound on the number of iterations. Coja-Oghlan [2010] – true for Belief Propagation for K-SAT. • Establish limits for the performance of local algorithms for random constraint satisfaction problems using the most general (Computer Science) definition of local algorithms.
  31. 31. Mick can’t get satisfaction above (from Sequential Local Algorithms) Chvatal & Reed [1998] ’’Mick gets some (the odds are on his side) [satisfiability]’’ Parting thoughts …
  32. 32. Thank you
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×