Upcoming SlideShare
×

# Counting Independent Sets using BP for Sparse Graphs ...

400 views

Published on

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
400
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
6
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Counting Independent Sets using BP for Sparse Graphs ...

1. 1. Counting Independent Sets using BP for Sparse Graphs Computing the explicit bound of loop series Michael Chertkov1 Devavrat Shah2 Jinwoo Shin2 1 Theory Division, LANL 2 LIDS, Massachusetts Institute of Technology October 14, 2008 DIMACS, Rutgers
2. 2. Introduction Our Results Conclusion Counting Independent Set Deﬁnition (Independent set) For a given graph G = (V , E ) with |V | = n, S ⊂ 2V is an independent set if ∀i , j ∈ S, (i , j) ∈ E . / Counting the number Z of independent sets Valiant 1979 - #P-complete Weitz 2006, Bandyopadhya, Gamarnik 2006 - (1 + ε) approximation algorithm for Z (or ln Z ) when a max-degree d ≤ 5. Bandyopadhya, Gamarnik 2006 - when G is a random d-regular graph and d ≤ 5, ∃ a constant αd s.t. 1 ln Z −→ αd w.h.p. n n→∞ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
3. 3. Introduction Our Results Conclusion Outline 1 Introduction Summary of Results Belief Propagation Loop Series 2 Our Results Result I: Loop Series for large girth graphs Result II: Loop Series for 3-random regular graphs 3 Conclusion Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
4. 4. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result I Let ZBP be the value BP estimated for Z . When G has a large girth g > cd log2 n with cd = 8d, 1 Z = ZBP 1 ± O . n Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
5. 5. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result I Let ZBP be the value BP estimated for Z . When G has a large girth g > cd log2 n with cd = 8d, 1 Z = ZBP 1 ± O . n If Cycle Double Cover Conjecture (Szekeres, Seymour 1970’) is true, cd = 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
6. 6. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
7. 7. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
8. 8. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. O(1)-error is unexpected in Stat. Physics. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
9. 9. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Our Results Result II If Shortest Cycle Cover Conjecture (Alon, Tarsi 1985) is true and G is a random 3-regular graph with n vertices, ln Z = α3 n ± O(1) w.h.p, where α3 = ln 1.545 . . . . α3 is obtained by solving the BP equation. Recall Bandyopadhya, Gamarnik 2006 implies ln Z = α3 n ± o(n) w.h.p. O(1)-error is unexpected in Stat. Physics. Jamshy and Tarsi 1992 - SCCC implies CDCC. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
10. 10. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
11. 11. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.50 0.50 0.50 0.50 0.50 0.50 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
12. 12. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.66 0.66 0.66 0.66 0.66 0.66 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
13. 13. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.60 0.60 0.60 0.60 0.60 0.60 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
14. 14. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.63 0.63 0.63 0.63 0.63 0.63 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
15. 15. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Belief Propagation Algorithm for counting # independent sets 1. Initialize: mu→v (0) ← 1 . 2 2. Update: 1 mu→v (t + 1) ← . 1+ w∈N (u)v mw→u (t) 0.62 0.62 0.62 0.62 0.62 0.62 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
16. 16. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series BP Estimations Marginal probability τv (1) = mw→v τv (0) + τv (1) = 1 τv (0) w∈N (v) τu,v (0, 1) = τv (1) τu,v (1, 0) = τu (1) τu,v (1, 1) = 0 τu,v (0, 0) = 1 − τv (1) − τu (1) Partition function Z ln ZBP = H(τv ) − I (τv : τu ) v∈V (u,v)∈E = (−xv ln xv + (dv − 1)(1 − xv ) ln(1 − xv )) v∈V − (1 − xu − xv ) ln(1 − xu − xv ) (xv = τv (1)) (u,v)∈E Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
17. 17. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Quality of BP: Loop Series Convergence BP ﬁxed points always exist from the Brouwer ﬁxed point theorem. The convergence to ﬁxed points is not guaranteed in loopy graphs. Fixed points can be achievable via Bethe variational method. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
18. 18. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Quality of BP: Loop Series Convergence BP ﬁxed points always exist from the Brouwer ﬁxed point theorem. The convergence to ﬁxed points is not guaranteed in loopy graphs. Fixed points can be achievable via Bethe variational method. Quality of Estimation: Loop Series (Chernyak, Chertkov 2006)   Z = ZBP 1 + w (F ) , ∅=F ⊂E where dF (v)−1 τv (1) w (F ) = (−1)|F | τv (1) 1 + (−1)dF (v) . v∈VF τv (0) If ∃v with dF (v ) = 1, then w (F ) = 0. If ∄v with dF (v ) = 1, F is called the generalized loop. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
19. 19. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Recent works on Loop Series Sudderth, Wainwright, Willsky 2007 - provide the relation of the loop series to the ”tree reparametrization” concept. Chandrasekhar, Gamarnik and Shah 2008 - provide bounds between ln Z and ln ZBP for a class of MRF using loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
20. 20. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
21. 21. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
22. 22. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Therefore, Z = ZBP (1 + w (F )) = 4.24(1 − 0.056) = 4. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
23. 23. Introduction Summary of Results Our Results Belief Propagation Conclusion Loop Series Example: Loop Series of a triangle K3 Calculating ZBP Solving BP equations 1 - 6 variables (mu→v ) and 6 equations mu→v = 1+ mw →u . w ∈N (u)v 1 - mu→v = x, where x = 1+x = 0.618 . . . . τv (1) 2 τv (0) = x = 0.382 → τv (1) = 0.276. ZBP = 4.24. Calculating 1 + w (F ) K3 itself is only a generalized loop F . τv (1) w (F ) = − v∈VF τv (1) 1 + τv (0) = −x 6 = −0.056 Therefore, Z = ZBP (1 + w (F )) = 4.24(1 − 0.056) = 4. In general, 1 + w (F ) is hard to compute since there are exponentially many generalized loops! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
24. 24. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Loop Series for large girth graphs Theorem If the girth of G is greater than cd log2 n with cd = 8d, 1 |w (F )| = O . n ∅=F ⊂E Corollary If the girth of G is greater than cd log2 n, 1 Z = ZBP 1 ± O . n Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
25. 25. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? How many generalized loops of size k? Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
26. 26. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP ﬁxed points. How many generalized loops of size k? Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
27. 27. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP ﬁxed points. How many generalized loops of size k? Want the number < γ k with small γ > 1. Need to count eﬃciently. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
28. 28. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Main Issues How can we bound |w (F )|? Issues What is |w (F )| for a generalized loop F of size k? Want |w (F )| < β k with small β < 1. Need to analyze BP ﬁxed points. How many generalized loops of size k? Want the number < γ k with small γ > 1. Need to count eﬃciently. Main observation If βγ < 1, |w (F )| becomes a geometric decaying series starting from the girth. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
29. 29. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Computing Strategy using Apples Deﬁnition (Apple) A connected subgraph C of G is an apple if one of the followings satisﬁes. C is a cycle. C is the union of a cycle and a line. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
30. 30. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Computing Strategy using Apples Deﬁnition (Apple) A connected subgraph C of G is an apple if one of the followings satisﬁes. C is a cycle. C is the union of a cycle and a line. Strategy for bounding the loop series 1. Decompose the sum 1 + |w (F )| into the product of apples-terms. ? 1+ |w (F )| = (1 + |w (C )|) . F C 2. Analyze the number and weights of apples. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
31. 31. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
32. 32. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
33. 33. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
34. 34. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i If any F has its decomposition {Ci } with apple-vertex degree ≤ T , 1 1 1+ |w (F )| < 1 + |w (C )| T < exp (γβ T )k . F C k Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
35. 35. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Decompose Sum into Product Assumption |w (C )| < β k for a apple C of size k The number of apples of size k is at most γ k . How can we bound |w (F )| using this information? If F = ∪Ci and Ci are disjoint, w (F ) = i w (Ci ). If F = ∪Ci with apple-vertex degree T = maxv∈F |{Ci : v ∈ Ci }|, 1 |w (F )| < |w (Ci )| T . i If any F has its decomposition {Ci } with apple-vertex degree ≤ T , 1 1 1 1+ |w (F )| < 1 + |w (C )| T < exp (γβ T )k . Want γβ T < 1 !! F C k Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
36. 36. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
37. 37. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 The number of apples of size k Each apple of size k corresponds to a k-level leaf of the self avoiding tree. The number of apples of size k is at most γ k where γ is the expansion rate of the tree. As the girth grows, γ → 1. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
38. 38. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Apples The weight w (C ) of apples of size k Easy to analyze due to the simple structure of apples. We obtained |w (C )| < β k with β = 1 . 2 The number of apples of size k Each apple of size k corresponds to a k-level leaf of the self avoiding tree. The number of apples of size k is at most γ k where γ is the expansion rate of the tree. As the girth grows, γ → 1. 1 Recall we want γβ T < 1, hence if we ﬁnd a constant T , we are done! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
39. 39. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
40. 40. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Theorem (Bermond, Jackson and Jaeger 1983) For every biconnected (bridgeless) graph, there exists a list of cycles so that every edge is contained in exactly four cycles of the list. The case of two is called the Cycle Double Cover Conjecture. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
41. 41. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result I: Analysis of Decomposition Quality Theorem For any generalized loop F , there exists its decomposition into apples with apple-vertex degree T ≤ 4d. Theorem (Bermond, Jackson and Jaeger 1983) For every biconnected (bridgeless) graph, there exists a list of cycles so that every edge is contained in exactly four cycles of the list. The case of two is called the Cycle Double Cover Conjecture. Decomposition Strategy 1 Decompose F into biconnected components with connecting edges. 2 From Lemma, cover each component with cycles. 3 Cover the remaining connecting edges by making cycles to be apples. 4 Apple-edge degree 4 leads to apple-vertex degree 4d. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
42. 42. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Loop Series for 3-random regular graphs Theorem If the Shortest Cycle Cover Conjecture is true and G is a random 3-regular graph with n vertices, there exists a ﬁnite-valued function f : (0, 1) → R+ such that ln Z = ln ZBP ± f (ε) with probability 1 − ε, where ln ZBP = α3 n and α3 = ln 1.545 . . . . α3 is obtained from the homogeneous solution of BP equation. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
43. 43. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Loop Series for 3-random regular graphs Theorem If the Shortest Cycle Cover Conjecture is true and G is a random 3-regular graph with n vertices, there exists a ﬁnite-valued function f : (0, 1) → R+ such that ln Z = ln ZBP ± f (ε) with probability 1 − ε, where ln ZBP = α3 n and α3 = ln 1.545 . . . . α3 is obtained from the homogeneous solution of BP equation. Shortest Cycle Cover Conjecture (Alon and Tarsi 1985) The edges of every biconnected graph with m edges can be covered by cycles of total length at most 7m/5 = 1.4m. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
44. 44. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Upper bound Properties of Random 3-regular graphs The number of cycles of size k is ≤ 2k /k. The number of apples of size k is 2k , hence γ = 2. 1 → If SCCC is true, γβ T ≈ 0.96 < 1. → |w (F )| = O(1). 1 (Recall |w (F )| = O( n ) in the large girth case.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
45. 45. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Upper bound Properties of Random 3-regular graphs The number of cycles of size k is ≤ 2k /k. The number of apples of size k is 2k , hence γ = 2. 1 → If SCCC is true, γβ T ≈ 0.96 < 1. → |w (F )| = O(1). 1 (Recall |w (F )| = O( n ) in the large girth case.) Upper bound of ln Z From loop calculus, ln Z = ln ZBP + ln(1 + w (F )) How can we get the ≤ ln ZBP + ln(1 + |w (F )|) lower bound? = ln ZBP + O(1). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
46. 46. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
47. 47. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. However, now |w (F )| = O(1) > 1 !! Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
48. 48. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Lower bound Why was it possible in the large girth case? Since |w (F )| = O( 1 ) < 0.5, n Z = ZBP 1 + w (F ) > ZBP 1 − |w (F )| > ZBP /2. However, now |w (F )| = O(1) > 1 !! Main idea Find a new graph G ′ such that |w ′ (F )| < 0.5 but ln Z ≈ ln Z ′ ′ and ln ZBP ≈ ln ZBP . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
49. 49. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ′ ln ZBP ≈ ln ZBP ln Z ≈ ln Z ′ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
50. 50. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
51. 51. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
52. 52. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
53. 53. Introduction Result I: Loop Series for large girth graphs Our Results Result II: Loop Series for 3-random regular graphs Conclusion Result II: Construction of G ′ Basic Idea Reduce |w (F )| ← Break small cycles. ′ ln ZBP ≈ ln ZBP ← Keep regularity. ln Z ≈ ln Z ′ ← |V | ≈ |V ′ |. Algorithm to construct G ′ 1. G ′ ← G 2. While |w ′ (F )| < 0.5 3. Find the smallest cycle C in G ′ 4. Insert Hε on one of the edges of C (Hε should have a large girth.) Lemma Given ε > 0, there exists a 3-regular large-girth graph Hε such that F ⊂Hε |w (F )| < ε. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
54. 54. Introduction Our Results Conclusion Conclusion Beyond Correlation Decay The loop series is a new analytic tool. Techniques to bound the loop series Graphs with large girth: Decompose Sum into Product using apples. Random 3-regular graphs: Graph-Insertion for reducing the loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
55. 55. Introduction Our Results Conclusion Conclusion Beyond Correlation Decay The loop series is a new analytic tool. Techniques to bound the loop series Graphs with large girth: Decompose Sum into Product using apples. Random 3-regular graphs: Graph-Insertion for reducing the loop series. Future works Other graphical models - k-SAT, coloring, Ising, etc. Find bigger regimes with more errors: ln Z = ln ZBP ± o(1) −→ ln Z = ln ZBP ± o(n). how conditions go? Algorithmic implementation for reducing the loop series. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
56. 56. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
57. 57. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + K |Hε | ln 2 ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + K |Hε |α3 . Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
58. 58. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + f (ε) ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + f (ε). Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
59. 59. Introduction Our Results Conclusion Result II: Analysis of Algorithm In each iteration, |w ′ (F )| keeps decreasing if we choose a small ε. ⇒ Terminate in ﬁnite K steps since |w (F )| = O(1). G ′ becomes a bigger 3-regular graph with |Hε | more vertices. ⇒ ln Z ′ , ln ZBP increases by at most |Hε | ln 2, |Hε |α3 . ′ Finally, ln Z ≤ ln Z ′ ≤ ln Z + f (ε) ′ ln ZBP ≤ ln ZBP ≤ ln ZBP + f (ε). Since |w ′ (F )| < 0.5, ln Z ′ > ln ZBP − ln 2 ′ ⇒ ln Z > ln ZBP − f (ε) − ln 2. Jinwoo Shin, MIT Counting Independent Sets using BP for Sparse Graphs
60. 60. Introduction Our Results Conclusion Result II: Proof of Lemma Theorem (McKay, Wormald, Wysocka 2004) Xr = # cycles of length r in random 3-regular graph. E [Xr ] ≤ 2r /r = µr r → |w (F )| r Xr (0.48) := r ar with E [ar ] (0.96)r . X3 , X4 , . . . , X 1 log n are independent poisson r.v. with mean µr . 3 g → Pr [X3 = X4 = · · · = Xg = 0] ≈ e −e . Set g = log log log n. Probabilistic Analysis A1 = ar 1 r <g E 1 : A1 = 0 w.p. log n A2 = g ≤r < 1 log n ar E 2 : A2 ≤ 2E [A2 ] w.p. 1 3 2 A3 = ar 1 1 g ≥ 3 log n E 3 : A3 ≤ (3 log n)E [A3 ] w.p. 1 − 3 log n Pr[E 1&E 2&E 3] > 0. A1 + A2 + A3 ≤ 2E [A2Shin, MITlog n)E [A3 ] → 0. Sets using BP for Sparse Graphs Jinwoo ] + (3 Counting Independent