## Just for you: FREE 60-day trial to the world’s largest digital library.

The SlideShare family just got bigger. Enjoy access to millions of ebooks, audiobooks, magazines, and more from Scribd.

Cancel anytime.Free with a 14 day trial from Scribd

- 1. . . The Chow-Liu algorithm based on the MDL with discreete and continuous variables Joe Suzuki Osaka University AIGM 2014, Paris Joe Suzuki (Osaka University) The Chow-Liu algorithm based on the MDL with discreete aAnIGdMcon2t0i1n4u,ouPsarvisariable1s / 26
- 2. The Chow-Liu Algorithm Chow-Liu P1; ;N: Probability of X(1); ; X(N) N ( 1) G = (V; E): Undirected Graph E := fg, V := f1; ;Ng (N 1), E := ffi ; jgji̸= j ; i ; j 2 Vg do E̸= fg 1. choose fi ; jg 2 E that maximizes I (i ; j) 2. remove fi ; jg from E 3. if no loop is generated, add fi ; jg to E Mutual Information of X(i); X(j): I (i ; j) := Σ x(i) Σ x(j) Pi ;j (x(i); x(j)) log Pi ;j (x(i); x(j)) Pi (x(j))Pi (x(i)) . Tree E s.t. Σ fi ;jg2E I (i ; j) ! max . .D(P1; ;NjjQ) ! min Joe Suzuki (Osaka University) The Chow-Liu algorithm based on the MDL with discreete aAnIGdMcon2t0i1n4u,ouPsarvisariable2s / 26
- 3. The Chow-Liu Algorithm Example Q(x(1); x(2); x(3); x(4)) = P1;2(x(1); x(2))P1;3(x(1); x(3))P1;4(x(1); x(4)) P1(x(1))P2(x(1)) P1(x(1))P3(x(1)) P1(x(1))P4(x(4)) P1(x(1))P2(x(2))P3(x(3))P4(x(4)) = P(x(1))P(x(2)jx(1))P(x(3)jx(1))P(x(4)jx(1)) i 1 1 2 1 2 3 j 2 3 3 4 4 4 I (i ; j) 12 10 8 6 4 2 j j 1 3 j j 2 4 j j 1 3 j j 2 4 j j 1 3 j j 2 4 j j 1 3 @@ j j 2 4 Joe Suzuki (Osaka University) The Chow-Liu algorithm based on the MDL with discreete aAnIGdMcon2t0i1n4u,ouPsarvisariable3s / 26
- 4. The Chow-Liu Algorithm Dendroid Distribution X(1); ; X(N): Discrete Random Variables V := f1; ;Ng E ffi ; jgji̸= j ; i ; j 2 Vg Q(x(1); ; x(N)jE) = Π fi ;jg2E Pi ;j (x(i); x(j)) Pi (x(i))Pj (x(j)) Π i2V Pi (x(i)) ; fPi (x(i))gi2V , fPi ;j (x(i); x(j))gi̸=j : from P1; ;N(x(1); ; x(N)) Joe Suzuki (Osaka University) The Chow-Liu algorithm based on the MDL with discreete aAnIGdMcon2t0i1n4u,ouPsarvisariable4s / 26
- 5. The Chow-Liu Algorithm Contribution . Starting from Data . .Learning rather than Approximation distribution P1; ;N data xn = f(x(1) i ; ; x(N) i )gni =1 . In any database, .. .some
- 6. elds are discrete and others continuous Joe Suzuki: A Construction of Bayesian Networks from Databases Based on an MDL Principle, UAI 1993 David Edwords, et. al: Selecting high-dimensional mixed graphical models using minimal AIC or BIC forests, BMC Informatics 2010 Joe Suzuki: Learning Bayesian network structures when discrete and continous variables are present, PGM 2014 Joe Suzuki (Osaka University) The Chow-Liu algorithm based on the MDL with discreete aAnIGdMcon2t0i1n4u,ouPsarvisariable5s / 26
- 7. The Chow-Liu Algorithm Maximum Likelihood (ML) f^P i (x(i))gi2V , f^P i ;j (x(i); x(j))gi̸=j are obtained from xn ML Estimation of MI: ^I (i ; j) := Σ x(i) Σ x(j) ^P i ;j (x(i); x(j)) log ^P i ;j (x(i); x(j)) ^P i (x(j))^P i (x(i)) Empirical Entropy given E (minus Likelihood given E): ^H n(xnjE) := n Σ i2V ^H (i )