Counting Independent Sets using BP for Sparse Graphs ...

400 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
400
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Counting Independent Sets using BP for Sparse Graphs ...

  1. 1. Counting Independent Sets using BP for Sparse Graphs Computing the explicit bound of loop series Michael Chertkov1 Devavrat Shah2 Jinwoo Shin2 1 Theory Division, LANL 2 LIDS, Massachusetts Institute of Technology October 14, 2008 DIMACS, Rutgers
  2. 2. Introduction Our Results Conclusion Counting Independent Set Definition (Independent set) For a given graph G = (V , E ) with |V | = n, S ⊂ 2V is an independent set if ∀i , j ∈ S, (i , j) ∈ E . / Counting the number Z of independent sets Valiant 1979 - #P-complete Weitz 2006, Bandyopadhya, Gamarnik 2006 - (1 + ε) approximation algorithm for Z (or ln Z ) when a max-degree d ≤ 5. Bandyopadhya, Gamarnik 2006 - when G is a random d-regular graph and d ≤ 5, ∃ a constant αd s.t. 1 ln Z −→ αd w.h.p. n n→∞ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  3. 3. Introduction Our Results Conclusion Outline 1 Introduction Summary of Results Belief Propagation Loop Series 2 Our Results Result I: Loop Series for large girth graphs Result II: Loop Series for 3-random regular graphs 3 Conclusion Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  4. 4. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result I Let ZBP be the value BP estimated for Z . When G has a large girth g > cd log2 n with cd = 8d, 1 Z = ZBP 1 ± O . n Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  5. 5. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result I Let ZBP be the value BP estimated for Z . When G has a large girth g > cd log2 n with cd = 8d, 1 Z = ZBP 1 ± O . n If Cycle Double Cover Conjecture (Szekeres, Seymour 1970’) is true, cd = 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  6. 6. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  7. 7. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  8. 8. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. O(1)-error is unexpected in Stat. Physics. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  9. 9. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. O(1)-error is unexpected in Stat. Physics. Jamshy and Tarsi 1992 - SCCC implies CDCC. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  10. 10. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  11. 11. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.50 0.50 0.50 0.50 0.50 0.50 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  12. 12. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.66 0.66 0.66 0.66 0.66 0.66 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  13. 13. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.60 0.60 0.60 0.60 0.60 0.60 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  14. 14. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.63 0.63 0.63 0.63 0.63 0.63 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  15. 15. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.62 0.62 0.62 0.62 0.62 0.62 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  16. 16. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series BP Estimations Marginal probability τv (1) = mw→v τv (0) + τv (1) = 1 τv (0) w∈N (v) τu,v (0, 1) = τv (1) τu,v (1, 0) = τu (1) τu,v (1, 1) = 0 τu,v (0, 0) = 1 − τv (1) − τu (1) Partition function Z ln ZBP = H(τv ) − I (τv : τu ) v∈V (u,v)∈E = (−xv ln xv + (dv − 1)(1 − xv ) ln(1 − xv )) v∈V − (1 − xu − xv ) ln(1 − xu − xv ) (xv = τv (1)) (u,v)∈E Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  17. 17. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Quality of BP: Loop Series Convergence BP fixed points always exist from the Brouwer fixed point theorem. The convergence to fixed points is not guaranteed in loopy graphs. Fixed points can be achievable via Bethe variational method. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  18. 18. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Quality of BP: Loop Series Convergence BP fixed points always exist from the Brouwer fixed point theorem. The convergence to fixed points is not guaranteed in loopy graphs. Fixed points can be achievable via Bethe variational method. Quality of Estimation: Loop Series (Chernyak, Chertkov 2006)   Z = ZBP 1 + w (F ) , ∅=F ⊂E where dF (v)−1 τv (1) w (F ) = (−1)|F | τv (1) 1 + (−1)dF (v) . v∈VF τv (0) If ∃v with dF (v ) = 1, then w (F ) = 0. If ∄v with dF (v ) = 1, F is called the generalized loop. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  19. 19. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Recent works on Loop Series Sudderth, Wainwright, Willsky 2007 - provide the relation of the loop series to the ”tree reparametrization” concept. Chandrasekhar, Gamarnik and Shah 2008 - provide bounds between ln Z and ln ZBP for a class of MRF using loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  20. 20. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  21. 21. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  22. 22. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Therefore, Z = ZBP (1 + w (F )) = 4.24(1 − 0.056) = 4. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  23. 23. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Therefore, Z = ZBP (1 + w (F )) = 4.24(1 − 0.056) = 4. In general, 1 + w (F ) is hard to compute since there are exponentially many generalized loops! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  24. 24. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Loop Series for large girth graphs Theorem If the girth of G is greater than cd log2 n with cd = 8d, 1 |w (F )| = O . n ∅=F ⊂E Corollary If the girth of G is greater than cd log2 n, 1 Z = ZBP 1 ± O . n Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  25. 25. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? How many generalized loops of size k? Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  26. 26. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP fixed points. How many generalized loops of size k? Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  27. 27. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP fixed points. How many generalized loops of size k? Want the number < γ k with small γ > 1. Need to count efficiently. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  28. 28. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP fixed points. How many generalized loops of size k? Want the number < γ k with small γ > 1. Need to count efficiently. Main observation If βγ < 1, |w (F )| becomes a geometric decaying series starting from the girth. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  29. 29. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Computing Strategy using Apples Definition (Apple) A connected subgraph C of G is an apple if one of the followings satisfies. C is a cycle. C is the union of a cycle and a line. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  30. 30. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Computing Strategy using Apples Definition (Apple) A connected subgraph C of G is an apple if one of the followings satisfies. C is a cycle. C is the union of a cycle and a line. Strategy for bounding the loop series 1. Decompose the sum 1 + |w (F )| into the product of apples-terms. ? 1+ |w (F )| = (1 + |w (C )|) . F C 2. Analyze the number and weights of apples. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  31. 31. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  32. 32. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  33. 33. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  34. 34. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i If any F has its decomposition {Ci } with apple-vertex degree ≤ T , 1 1 1+ |w (F )| < 1 + |w (C )| T < exp (γβ T )k . F C k Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  35. 35. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i If any F has its decomposition {Ci } with apple-vertex degree ≤ T , 1 1 1 1+ |w (F )| < 1 + |w (C )| T < exp (γβ T )k . Want γβ T < 1 !! F C k Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  36. 36. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  37. 37. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 The number of apples of size k Each apple of size k corresponds to a k-level leaf of the self avoiding tree. The number of apples of size k is at most γ k where γ is the expansion rate of the tree. As the girth grows, γ → 1. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  38. 38. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 The number of apples of size k Each apple of size k corresponds to a k-level leaf of the self avoiding tree. The number of apples of size k is at most γ k where γ is the expansion rate of the tree. As the girth grows, γ → 1. 1 Recall we want γβ T < 1, hence if we find a constant T , we are done! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  39. 39. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  40. 40. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Theorem (Bermond, Jackson and Jaeger 1983) For every biconnected (bridgeless) graph, there exists a list of cycles so that every edge is contained in exactly four cycles of the list. The case of two is called the Cycle Double Cover Conjecture. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  41. 41. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Theorem (Bermond, Jackson and Jaeger 1983) For every biconnected (bridgeless) graph, there exists a list of cycles so that every edge is contained in exactly four cycles of the list. The case of two is called the Cycle Double Cover Conjecture. Decomposition Strategy 1 Decompose F into biconnected components with connecting edges. 2 From Lemma, cover each component with cycles. 3 Cover the remaining connecting edges by making cycles to be apples. 4 Apple-edge degree 4 leads to apple-vertex degree 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  42. 42. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Loop Series for 3-random regular graphs Theorem If the Shortest Cycle Cover Conjecture is true and G is a random 3-regular graph with n vertices, there exists a finite-valued function f : (0, 1) → R+ such that ln Z = ln ZBP ± f (ε) with probability 1 − ε, where ln ZBP = α3 n and α3 = ln 1.545 . . . . α3 is obtained from the homogeneous solution of BP equation. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  43. 43. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Loop Series for 3-random regular graphs Theorem If the Shortest Cycle Cover Conjecture is true and G is a random 3-regular graph with n vertices, there exists a finite-valued function f : (0, 1) → R+ such that ln Z = ln ZBP ± f (ε) with probability 1 − ε, where ln ZBP = α3 n and α3 = ln 1.545 . . . . α3 is obtained from the homogeneous solution of BP equation. Shortest Cycle Cover Conjecture (Alon and Tarsi 1985) The edges of every biconnected graph with m edges can be covered by cycles of total length at most 7m/5 = 1.4m. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  44. 44. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Upper bound Properties of Random 3-regular graphs The number of cycles of size k is ≤ 2k /k. The number of apples of size k is 2k , hence γ = 2. 1 → If SCCC is true, γβ T ≈ 0.96 < 1. → |w (F )| = O(1). 1 (Recall |w (F )| = O( n ) in the large girth case.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  45. 45. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Upper bound Properties of Random 3-regular graphs The number of cycles of size k is ≤ 2k /k. The number of apples of size k is 2k , hence γ = 2. 1 → If SCCC is true, γβ T ≈ 0.96 < 1. → |w (F )| = O(1). 1 (Recall |w (F )| = O( n ) in the large girth case.) Upper bound of ln Z From loop calculus, ln Z = ln ZBP + ln(1 + w (F )) How can we get the ≤ ln ZBP + ln(1 + |w (F )|) lower bound? = ln ZBP + O(1). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  46. 46. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  47. 47. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. However, now |w (F )| = O(1) > 1 !! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  48. 48. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. However, now |w (F )| = O(1) > 1 !! Main idea Find a new graph G ′ such that |w ′ (F )| < 0.5 but ln Z ≈ ln Z ′ ′ and ln ZBP ≈ ln ZBP . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  49. 49. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ′ ln ZBP ≈ ln ZBP ln Z ≈ ln Z ′ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  50. 50. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  51. 51. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  52. 52. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  53. 53. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Lemma Given ε > 0, there exists a 3-regular large-girth graph Hε such that F ⊂Hε |w (F )| < ε. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  54. 54. Introduction Our Results Conclusion Conclusion Beyond Correlation Decay The loop series is a new analytic tool. Techniques to bound the loop series Graphs with large girth: Decompose Sum into Product using apples. Random 3-regular graphs: Graph-Insertion for reducing the loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  55. 55. Introduction Our Results Conclusion Conclusion Beyond Correlation Decay The loop series is a new analytic tool. Techniques to bound the loop series Graphs with large girth: Decompose Sum into Product using apples. Random 3-regular graphs: Graph-Insertion for reducing the loop series. Future works Other graphical models - k-SAT, coloring, Ising, etc. Find bigger regimes with more errors: ln Z = ln ZBP ± o(1) −→ ln Z = ln ZBP ± o(n). how conditions go? Algorithmic implementation for reducing the loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  56. 56. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in finite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  57. 57. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in finite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + K |Hε | ln 2 ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + K |Hε |α3 . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  58. 58. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in finite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + f (ε) ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + f (ε). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  59. 59. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in finite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + f (ε) ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + f (ε). Since |w ′ (F )| < 0.5, ln Z ′ > ln ZBP − ln 2 ′ ⇒ ln Z > ln ZBP − f (ε) − ln 2. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
  60. 60. Introduction Our Results Conclusion Result II: Proof of Lemma Theorem (McKay, Wormald, Wysocka 2004) Xr = # cycles of length r in random 3-regular graph. E [Xr ] ≤ 2r /r = µr r → |w (F )| r Xr (0.48) := r ar with E [ar ] (0.96)r . X3 , X4 , . . . , X 1 log n are independent poisson r.v. with mean µr . 3 g → Pr [X3 = X4 = · · · = Xg = 0] ≈ e −e . Set g = log log log n. Probabilistic Analysis A1 = ar 1 r <g E 1 : A1 = 0 w.p. log n A2 = g ≤r < 1 log n ar E 2 : A2 ≤ 2E [A2 ] w.p. 1 3 2 A3 = ar 1 1 g ≥ 3 log n E 3 : A3 ≤ (3 log n)E [A3 ] w.p. 1 − 3 log n Pr[E 1&E 2&E 3] > 0. A1 + A2 + A3 ≤ 2E [A2Shin, MITlog n)E [A3 ] → 0. Sets using BP for Sparse Graphs Jinwoo ] + (3 Counting Independent

×