This document proposes STEPHY, a method for improving LDA topic models by allowing topic distributions to vary over time. STEPHY conducts multistage variational Bayesian inference over LDA models with increasing levels of time dependence, starting with a time-independent model and progressing to models where topic distributions change over timestamps. This multistage approach allows efficient inference by initializing parameters from the previous model. The method is evaluated on several datasets, showing improved perplexity over standard LDA.

1. ----- S teering ----- T ime-Dependent E stimation --- of P osteriors --- with HY perparameter Indexing - in Bayesian Topic Models - Tomonari MASADA ( 正田备也 ) Nagasaki University [email_address]

6. φ 1 φ K Multi( φ 1 ), Multi( φ 2 ), ... , Multi( φ K ) φ k =( φ k 1 , φ k 2, ..., φ kW ) LDA LDA

7. φ 1 φ K Di(β) β=(β 1 , β 2, ..., β W ) LDA LDA

8. φ 11 φ 1K φ TK φ T1

9. φ 11 φ 1K φ TK φ T1 K

10. φ 11 φ 1K φ TK φ T1 T

11. φ 11 φ 1K φ TK φ T1 Di(β) β=(β 1 , β 2, ..., β W ) Option 0 Option 0

12. φ 11 φ 1K φ TK φ T1 Option 1 Option 1 Di(β 1 ) . . . Di(β K ) β=(β k 1 , β k 2, ..., β kW )

13. φ 11 φ 1K φ TK φ T1 Option 2 Option 2 Di(β 1 ) . . . Di(β T ) β=(β t 1 , β t 2, ..., β tW )

14. φ 11 φ 1K φ TK φ T1 Option 3 Option 3 Di(β 11 ) . . . Di(β 1 K ) . . . . . . . . . Di(β T 1 ) . . . Di(β TK ) β=(β tk 1 , β tk 2, ..., β tkW )

15. PROPOSAL LDA Option 1 Option 3

16. ----- S teering ----- T ime-Dependent E stimation --- of P osteriors --- with HY perparameter Indexing - in Bayesian Topic Models - S T E P H Y

18. LDA Option 1 Option 3

20. DATA SPECS J W T P NIPS 1,740 11,998 13 919,916 DBLP 1,235,988 273,173 20 7,814,175 DONGA 24,093 71,621 53 7,949,288 TDT 96,256 51,849 123 11,460,231 NSF 128,181 25,325 13 10,388,976 YOMI 367,910 84,060 52 32,762,456

23. [Wang et al. 06]