Seminar Talk at Computational Intelligence Seminar F, Technical University Graz

1. Computational Intelligence Seminar F Topic Models LDA and the Correlated Topic Models Claudia Wagner Graz, 21.1.2011

3. Topic Models Generative Models http://www.cs.umass.edu/~wallach/talks/priors.pdf

5. Latent Dirichlet Allocation (LDA) (Blei et al, 2003)

7. Matrix Representation of LDA observed latent latent θ φ

8. Dirichlet Distribution

9. Dirichlet Distribution The larger the value of the concentration parameter alpha, the more evenly distributed is the resulting distribution! Low α Topic1 Topic 2 Topic 3 Examples with K=3 http://www.cs.umass.edu/~wallach/talks/priors.pdf θ i = (1, 0, 0) High α High α θ i = (1/3, 1/3, 1/3) More smoothing Less smoothing Topic1 Topic 2 Topic 3 ?

16. Correlated Topic Models (CTM) (Blei et al, 2007)

20. Dirichlet versus L ogistic Normal http://www.cs.princeton.edu/~blei/modeling-science.pdf Dirichlet distribution (Paramter: postive K-dim vector α) Logistic Normal distribution (Paramter: postive K-dim vector μ , K x K co-variance Matrix Ʃ ) High α Low α High α unsymmetric symmetric symmetric diagonal covariance negative correlation between topics 1 and 2 positive correlation between topics 1 and 2 Topic1 Topic2 Topic3

24. Empirical Results Comparing CTM and LDA (Blei et al, 2007)