Upcoming SlideShare
×

Counting Independent Sets using BP for Sparse Graphs ...

• 183 views

• Comment goes here.
Are you sure you want to
Be the first to comment
Be the first to like this

Total Views
183
On Slideshare
0
From Embeds
0
Number of Embeds
0

Shares
2
0
Likes
0

No embeds

Report content

No notes for slide

Transcript

• 1. Counting Independent Sets using BP for Sparse Graphs Computing the explicit bound of loop series Michael Chertkov1 Devavrat Shah2 Jinwoo Shin2 1 Theory Division, LANL 2 LIDS, Massachusetts Institute of Technology October 14, 2008 DIMACS, Rutgers
• 2. Introduction Our Results Conclusion Counting Independent Set Deﬁnition (Independent set) For a given graph G = (V , E ) with |V | = n, S ⊂ 2V is an independent set if ∀i , j ∈ S, (i , j) ∈ E . / Counting the number Z of independent sets Valiant 1979 - #P-complete Weitz 2006, Bandyopadhya, Gamarnik 2006 - (1 + ε) approximation algorithm for Z (or ln Z ) when a max-degree d ≤ 5. Bandyopadhya, Gamarnik 2006 - when G is a random d-regular graph and d ≤ 5, ∃ a constant αd s.t. 1 ln Z −→ αd w.h.p. n n→∞ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 3. Introduction Our Results Conclusion Outline 1 Introduction Summary of Results Belief Propagation Loop Series 2 Our Results Result I: Loop Series for large girth graphs Result II: Loop Series for 3-random regular graphs 3 Conclusion Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 4. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result I Let ZBP be the value BP estimated for Z . When G has a large girth g > cd log2 n with cd = 8d, 1 Z = ZBP 1 ± O . n Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 5. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result I Let ZBP be the value BP estimated for Z . When G has a large girth g > cd log2 n with cd = 8d, 1 Z = ZBP 1 ± O . n If Cycle Double Cover Conjecture (Szekeres, Seymour 1970’) is true, cd = 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 6. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 7. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 8. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. O(1)-error is unexpected in Stat. Physics. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 9. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. O(1)-error is unexpected in Stat. Physics. Jamshy and Tarsi 1992 - SCCC implies CDCC. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 10. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 11. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.50 0.50 0.50 0.50 0.50 0.50 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 12. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.66 0.66 0.66 0.66 0.66 0.66 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 13. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.60 0.60 0.60 0.60 0.60 0.60 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 14. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.63 0.63 0.63 0.63 0.63 0.63 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 15. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.62 0.62 0.62 0.62 0.62 0.62 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 16. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series BP Estimations Marginal probability τv (1) = mw→v τv (0) + τv (1) = 1 τv (0) w∈N (v) τu,v (0, 1) = τv (1) τu,v (1, 0) = τu (1) τu,v (1, 1) = 0 τu,v (0, 0) = 1 − τv (1) − τu (1) Partition function Z ln ZBP = H(τv ) − I (τv : τu ) v∈V (u,v)∈E = (−xv ln xv + (dv − 1)(1 − xv ) ln(1 − xv )) v∈V − (1 − xu − xv ) ln(1 − xu − xv ) (xv = τv (1)) (u,v)∈E Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 17. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Quality of BP: Loop Series Convergence BP ﬁxed points always exist from the Brouwer ﬁxed point theorem. The convergence to ﬁxed points is not guaranteed in loopy graphs. Fixed points can be achievable via Bethe variational method. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 18. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Quality of BP: Loop Series Convergence BP ﬁxed points always exist from the Brouwer ﬁxed point theorem. The convergence to ﬁxed points is not guaranteed in loopy graphs. Fixed points can be achievable via Bethe variational method. Quality of Estimation: Loop Series (Chernyak, Chertkov 2006)   Z = ZBP 1 + w (F ) , ∅=F ⊂E where dF (v)−1 τv (1) w (F ) = (−1)|F | τv (1) 1 + (−1)dF (v) . v∈VF τv (0) If ∃v with dF (v ) = 1, then w (F ) = 0. If ∄v with dF (v ) = 1, F is called the generalized loop. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 19. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Recent works on Loop Series Sudderth, Wainwright, Willsky 2007 - provide the relation of the loop series to the ”tree reparametrization” concept. Chandrasekhar, Gamarnik and Shah 2008 - provide bounds between ln Z and ln ZBP for a class of MRF using loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 20. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 21. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 22. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Therefore, Z = ZBP (1 + w (F )) = 4.24(1 − 0.056) = 4. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 23. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Therefore, Z = ZBP (1 + w (F )) = 4.24(1 − 0.056) = 4. In general, 1 + w (F ) is hard to compute since there are exponentially many generalized loops! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 24. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Loop Series for large girth graphs Theorem If the girth of G is greater than cd log2 n with cd = 8d, 1 |w (F )| = O . n ∅=F ⊂E Corollary If the girth of G is greater than cd log2 n, 1 Z = ZBP 1 ± O . n Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 25. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? How many generalized loops of size k? Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 26. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP ﬁxed points. How many generalized loops of size k? Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 27. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP ﬁxed points. How many generalized loops of size k? Want the number < γ k with small γ > 1. Need to count eﬃciently. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 28. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP ﬁxed points. How many generalized loops of size k? Want the number < γ k with small γ > 1. Need to count eﬃciently. Main observation If βγ < 1, |w (F )| becomes a geometric decaying series starting from the girth. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 29. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Computing Strategy using Apples Deﬁnition (Apple) A connected subgraph C of G is an apple if one of the followings satisﬁes. C is a cycle. C is the union of a cycle and a line. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 30. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Computing Strategy using Apples Deﬁnition (Apple) A connected subgraph C of G is an apple if one of the followings satisﬁes. C is a cycle. C is the union of a cycle and a line. Strategy for bounding the loop series 1. Decompose the sum 1 + |w (F )| into the product of apples-terms. ? 1+ |w (F )| = (1 + |w (C )|) . F C 2. Analyze the number and weights of apples. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 31. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 32. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 33. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 34. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i If any F has its decomposition {Ci } with apple-vertex degree ≤ T , 1 1 1+ |w (F )| < 1 + |w (C )| T < exp (γβ T )k . F C k Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 35. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i If any F has its decomposition {Ci } with apple-vertex degree ≤ T , 1 1 1 1+ |w (F )| < 1 + |w (C )| T < exp (γβ T )k . Want γβ T < 1 !! F C k Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 36. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 37. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 The number of apples of size k Each apple of size k corresponds to a k-level leaf of the self avoiding tree. The number of apples of size k is at most γ k where γ is the expansion rate of the tree. As the girth grows, γ → 1. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 38. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 The number of apples of size k Each apple of size k corresponds to a k-level leaf of the self avoiding tree. The number of apples of size k is at most γ k where γ is the expansion rate of the tree. As the girth grows, γ → 1. 1 Recall we want γβ T < 1, hence if we ﬁnd a constant T , we are done! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 39. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 40. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Theorem (Bermond, Jackson and Jaeger 1983) For every biconnected (bridgeless) graph, there exists a list of cycles so that every edge is contained in exactly four cycles of the list. The case of two is called the Cycle Double Cover Conjecture. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 41. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Theorem (Bermond, Jackson and Jaeger 1983) For every biconnected (bridgeless) graph, there exists a list of cycles so that every edge is contained in exactly four cycles of the list. The case of two is called the Cycle Double Cover Conjecture. Decomposition Strategy 1 Decompose F into biconnected components with connecting edges. 2 From Lemma, cover each component with cycles. 3 Cover the remaining connecting edges by making cycles to be apples. 4 Apple-edge degree 4 leads to apple-vertex degree 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 42. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Loop Series for 3-random regular graphs Theorem If the Shortest Cycle Cover Conjecture is true and G is a random 3-regular graph with n vertices, there exists a ﬁnite-valued function f : (0, 1) → R+ such that ln Z = ln ZBP ± f (ε) with probability 1 − ε, where ln ZBP = α3 n and α3 = ln 1.545 . . . . α3 is obtained from the homogeneous solution of BP equation. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 43. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Loop Series for 3-random regular graphs Theorem If the Shortest Cycle Cover Conjecture is true and G is a random 3-regular graph with n vertices, there exists a ﬁnite-valued function f : (0, 1) → R+ such that ln Z = ln ZBP ± f (ε) with probability 1 − ε, where ln ZBP = α3 n and α3 = ln 1.545 . . . . α3 is obtained from the homogeneous solution of BP equation. Shortest Cycle Cover Conjecture (Alon and Tarsi 1985) The edges of every biconnected graph with m edges can be covered by cycles of total length at most 7m/5 = 1.4m. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 44. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Upper bound Properties of Random 3-regular graphs The number of cycles of size k is ≤ 2k /k. The number of apples of size k is 2k , hence γ = 2. 1 → If SCCC is true, γβ T ≈ 0.96 < 1. → |w (F )| = O(1). 1 (Recall |w (F )| = O( n ) in the large girth case.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 45. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Upper bound Properties of Random 3-regular graphs The number of cycles of size k is ≤ 2k /k. The number of apples of size k is 2k , hence γ = 2. 1 → If SCCC is true, γβ T ≈ 0.96 < 1. → |w (F )| = O(1). 1 (Recall |w (F )| = O( n ) in the large girth case.) Upper bound of ln Z From loop calculus, ln Z = ln ZBP + ln(1 + w (F )) How can we get the ≤ ln ZBP + ln(1 + |w (F )|) lower bound? = ln ZBP + O(1). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 46. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 47. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. However, now |w (F )| = O(1) > 1 !! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 48. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. However, now |w (F )| = O(1) > 1 !! Main idea Find a new graph G ′ such that |w ′ (F )| < 0.5 but ln Z ≈ ln Z ′ ′ and ln ZBP ≈ ln ZBP . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 49. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ′ ln ZBP ≈ ln ZBP ln Z ≈ ln Z ′ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 50. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 51. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 52. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 53. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Lemma Given ε > 0, there exists a 3-regular large-girth graph Hε such that F ⊂Hε |w (F )| < ε. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 54. Introduction Our Results Conclusion Conclusion Beyond Correlation Decay The loop series is a new analytic tool. Techniques to bound the loop series Graphs with large girth: Decompose Sum into Product using apples. Random 3-regular graphs: Graph-Insertion for reducing the loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 55. Introduction Our Results Conclusion Conclusion Beyond Correlation Decay The loop series is a new analytic tool. Techniques to bound the loop series Graphs with large girth: Decompose Sum into Product using apples. Random 3-regular graphs: Graph-Insertion for reducing the loop series. Future works Other graphical models - k-SAT, coloring, Ising, etc. Find bigger regimes with more errors: ln Z = ln ZBP ± o(1) −→ ln Z = ln ZBP ± o(n). how conditions go? Algorithmic implementation for reducing the loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 56. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 57. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + K |Hε | ln 2 ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + K |Hε |α3 . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 58. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + f (ε) ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + f (ε). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 59. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + f (ε) ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + f (ε). Since |w ′ (F )| < 0.5, ln Z ′ > ln ZBP − ln 2 ′ ⇒ ln Z > ln ZBP − f (ε) − ln 2. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
• 60. Introduction Our Results Conclusion Result II: Proof of Lemma Theorem (McKay, Wormald, Wysocka 2004) Xr = # cycles of length r in random 3-regular graph. E [Xr ] ≤ 2r /r = µr r → |w (F )| r Xr (0.48) := r ar with E [ar ] (0.96)r . X3 , X4 , . . . , X 1 log n are independent poisson r.v. with mean µr . 3 g → Pr [X3 = X4 = · · · = Xg = 0] ≈ e −e . Set g = log log log n. Probabilistic Analysis A1 = ar 1 r <g E 1 : A1 = 0 w.p. log n A2 = g ≤r < 1 log n ar E 2 : A2 ≤ 2E [A2 ] w.p. 1 3 2 A3 = ar 1 1 g ≥ 3 log n E 3 : A3 ≤ (3 log n)E [A3 ] w.p. 1 − 3 log n Pr[E 1&E 2&E 3] > 0. A1 + A2 + A3 ≤ 2E [A2Shin, MITlog n)E [A3 ] → 0. Sets using BP for Sparse Graphs Jinwoo ] + (3 Counting Independent