SlideShare a Scribd company logo
1 of 24
Modeling Topical Trends over Continuous Time with Priors Tomonari MASADA  正田备也 Nagasaki University  长崎大学 [email_address] Tomonari MASADA ISNN 2010
Background ,[object Object],[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
Dynamism in Topical Trends ,[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
Latent Dirichlet Allocation [Blei et al. 03] ,[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
Tomonari MASADA ISNN 2010 Shanghai is the largest city in China, located in her eastern coast at the outlet of the Yangtze River. Originally a fishing and textiles town, Shanghai grew to importance in the 19th century. In 2005 Shanghai became the world's busiest cargo port. The city is an emerging tourist destination renowned for its historical landmarks such as the Bund and Xintiandi, its modern and 63 7 7 41 63 7 63 41 7 7 41 41 7 7 41 7 7 63 22 41 7 22 7 41 41 63 41 7 41 7 7 50 50 7 50 63 41 7 41 41 22 22 7 41 7 7 41 41 41 41 7 7 41 41 7 7 7 63 7 63 7 41 7
Topic Re-assignment Probabilities Tomonari MASADA ISNN 2010 How many tokens of word  w are assigned to topic  k How many tokens in doc  j are assigned to topic  k Introduce time-dependency!
Topics over Time [Wang et al. KDD06] Tomonari MASADA ISNN 2010
Tomonari MASADA ISNN 2010 TOT Beta densities
Tomonari MASADA ISNN 2010 LDA: no time-dependency TOT: too heavy time-dependency
LYNDA   [Masada et al. CIKM09] L atent d YN amical  D irichlet  A llocation Tomonari MASADA ISNN 2010
Tomonari MASADA ISNN 2010 LYNDA unnormalized Gaussian densities
Bayesian TOT   [this paper] ,[object Object],[object Object],Tomonari MASADA ISNN 2010
Use Gamma as Conjugate Prior ,[object Object],Tomonari MASADA ISNN 2010
x y z θ α φ s τ η γ β b a
Evaluation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
Tomonari MASADA ISNN 2010 entire document set ground-truth document set “ Yes!” “ No!”
TDT4 Dataset ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010 Weighting Words with Predictive Probabilities
Interpreting  Weighting Scheme ,[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
Summary of Evaluation ,[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010 ,[object Object],[object Object],[object Object],[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],Tomonari MASADA ISNN 2010
Tomonari MASADA ISNN 2010 http://www.cis.nagasaki-u.ac.jp/~masada/researches.html DBLP 1990 ~ 2009 LDA BTOT
Tomonari MASADA ISNN 2010 http://www.cis.nagasaki-u.ac.jp/~masada/researches.html Xinhua net   May 5, 2009 ~ Dec. 17, 2009 LDA BTOT
Tomonari MASADA ISNN 2010 http://www.cis.nagasaki-u.ac.jp/~masada/researches.html Yomiuri Newspaper 2002 ~ 2005 LDA BTOT

More Related Content

Viewers also liked

Supplementary material for my following paper: Infinite Latent Process Decomp...
Supplementary material for my following paper: Infinite Latent Process Decomp...Supplementary material for my following paper: Infinite Latent Process Decomp...
Supplementary material for my following paper: Infinite Latent Process Decomp...
Tomonari Masada
 
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet AllocationA Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
Tomonari Masada
 

Viewers also liked (10)

ChronoSAGE: Diversifying Topic Modeling Chronologically
ChronoSAGE: Diversifying Topic Modeling ChronologicallyChronoSAGE: Diversifying Topic Modeling Chronologically
ChronoSAGE: Diversifying Topic Modeling Chronologically
 
A Safe Rule for Sparse Logistic Regression
A Safe Rule for Sparse Logistic RegressionA Safe Rule for Sparse Logistic Regression
A Safe Rule for Sparse Logistic Regression
 
Supplementary material for my following paper: Infinite Latent Process Decomp...
Supplementary material for my following paper: Infinite Latent Process Decomp...Supplementary material for my following paper: Infinite Latent Process Decomp...
Supplementary material for my following paper: Infinite Latent Process Decomp...
 
Linked Open Dataの基礎と これからの情報活用
Linked Open Dataの基礎と これからの情報活用Linked Open Dataの基礎と これからの情報活用
Linked Open Dataの基礎と これからの情報活用
 
NIPS2014読み会 NIPS参加報告
NIPS2014読み会 NIPS参加報告NIPS2014読み会 NIPS参加報告
NIPS2014読み会 NIPS参加報告
 
MIRU2013チュートリアル:SIFTとそれ以降のアプローチ
MIRU2013チュートリアル:SIFTとそれ以降のアプローチMIRU2013チュートリアル:SIFTとそれ以降のアプローチ
MIRU2013チュートリアル:SIFTとそれ以降のアプローチ
 
論文紹介 Semi-supervised Learning with Deep Generative Models
論文紹介 Semi-supervised Learning with Deep Generative Models論文紹介 Semi-supervised Learning with Deep Generative Models
論文紹介 Semi-supervised Learning with Deep Generative Models
 
Poisson factorization
Poisson factorizationPoisson factorization
Poisson factorization
 
A Simple Stochastic Gradient Variational Bayes for the Correlated Topic Model
A Simple Stochastic Gradient Variational Bayes for the Correlated Topic ModelA Simple Stochastic Gradient Variational Bayes for the Correlated Topic Model
A Simple Stochastic Gradient Variational Bayes for the Correlated Topic Model
 
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet AllocationA Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
 

More from Tomonari Masada

More from Tomonari Masada (20)

Learning Latent Space Energy Based Prior Modelの解説
Learning Latent Space Energy Based Prior Modelの解説Learning Latent Space Energy Based Prior Modelの解説
Learning Latent Space Energy Based Prior Modelの解説
 
Denoising Diffusion Probabilistic Modelsの重要な式の解説
Denoising Diffusion Probabilistic Modelsの重要な式の解説Denoising Diffusion Probabilistic Modelsの重要な式の解説
Denoising Diffusion Probabilistic Modelsの重要な式の解説
 
Context-dependent Token-wise Variational Autoencoder for Topic Modeling
Context-dependent Token-wise Variational Autoencoder for Topic ModelingContext-dependent Token-wise Variational Autoencoder for Topic Modeling
Context-dependent Token-wise Variational Autoencoder for Topic Modeling
 
A note on the density of Gumbel-softmax
A note on the density of Gumbel-softmaxA note on the density of Gumbel-softmax
A note on the density of Gumbel-softmax
 
トピックモデルの基礎と応用
トピックモデルの基礎と応用トピックモデルの基礎と応用
トピックモデルの基礎と応用
 
Expectation propagation for latent Dirichlet allocation
Expectation propagation for latent Dirichlet allocationExpectation propagation for latent Dirichlet allocation
Expectation propagation for latent Dirichlet allocation
 
Mini-batch Variational Inference for Time-Aware Topic Modeling
Mini-batch Variational Inference for Time-Aware Topic ModelingMini-batch Variational Inference for Time-Aware Topic Modeling
Mini-batch Variational Inference for Time-Aware Topic Modeling
 
A note on variational inference for the univariate Gaussian
A note on variational inference for the univariate GaussianA note on variational inference for the univariate Gaussian
A note on variational inference for the univariate Gaussian
 
Document Modeling with Implicit Approximate Posterior Distributions
Document Modeling with Implicit Approximate Posterior DistributionsDocument Modeling with Implicit Approximate Posterior Distributions
Document Modeling with Implicit Approximate Posterior Distributions
 
LDA-Based Scoring of Sequences Generated by RNN for Automatic Tanka Composition
LDA-Based Scoring of Sequences Generated by RNN for Automatic Tanka CompositionLDA-Based Scoring of Sequences Generated by RNN for Automatic Tanka Composition
LDA-Based Scoring of Sequences Generated by RNN for Automatic Tanka Composition
 
A Note on ZINB-VAE
A Note on ZINB-VAEA Note on ZINB-VAE
A Note on ZINB-VAE
 
A Note on Latent LSTM Allocation
A Note on Latent LSTM AllocationA Note on Latent LSTM Allocation
A Note on Latent LSTM Allocation
 
A Note on TopicRNN
A Note on TopicRNNA Note on TopicRNN
A Note on TopicRNN
 
Topic modeling with Poisson factorization (2)
Topic modeling with Poisson factorization (2)Topic modeling with Poisson factorization (2)
Topic modeling with Poisson factorization (2)
 
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet AllocationA Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
A Simple Stochastic Gradient Variational Bayes for Latent Dirichlet Allocation
 
FDSE2015
FDSE2015FDSE2015
FDSE2015
 
A derivation of the sampling formulas for An Entity-Topic Model for Entity Li...
A derivation of the sampling formulas for An Entity-Topic Model for Entity Li...A derivation of the sampling formulas for An Entity-Topic Model for Entity Li...
A derivation of the sampling formulas for An Entity-Topic Model for Entity Li...
 
A Note on BPTT for LSTM LM
A Note on BPTT for LSTM LMA Note on BPTT for LSTM LM
A Note on BPTT for LSTM LM
 
The detailed derivation of the derivatives in Table 2 of Marginalized Denoisi...
The detailed derivation of the derivatives in Table 2 of Marginalized Denoisi...The detailed derivation of the derivatives in Table 2 of Marginalized Denoisi...
The detailed derivation of the derivatives in Table 2 of Marginalized Denoisi...
 
A Note on PCVB0 for HDP-LDA
A Note on PCVB0 for HDP-LDAA Note on PCVB0 for HDP-LDA
A Note on PCVB0 for HDP-LDA
 

Modeling Topical Trends over Continuous Time with Priors

  • 1. Modeling Topical Trends over Continuous Time with Priors Tomonari MASADA 正田备也 Nagasaki University 长崎大学 [email_address] Tomonari MASADA ISNN 2010
  • 2.
  • 3.
  • 4.
  • 5. Tomonari MASADA ISNN 2010 Shanghai is the largest city in China, located in her eastern coast at the outlet of the Yangtze River. Originally a fishing and textiles town, Shanghai grew to importance in the 19th century. In 2005 Shanghai became the world's busiest cargo port. The city is an emerging tourist destination renowned for its historical landmarks such as the Bund and Xintiandi, its modern and 63 7 7 41 63 7 63 41 7 7 41 41 7 7 41 7 7 63 22 41 7 22 7 41 41 63 41 7 41 7 7 50 50 7 50 63 41 7 41 41 22 22 7 41 7 7 41 41 41 41 7 7 41 41 7 7 7 63 7 63 7 41 7
  • 6. Topic Re-assignment Probabilities Tomonari MASADA ISNN 2010 How many tokens of word w are assigned to topic k How many tokens in doc j are assigned to topic k Introduce time-dependency!
  • 7. Topics over Time [Wang et al. KDD06] Tomonari MASADA ISNN 2010
  • 8. Tomonari MASADA ISNN 2010 TOT Beta densities
  • 9. Tomonari MASADA ISNN 2010 LDA: no time-dependency TOT: too heavy time-dependency
  • 10. LYNDA [Masada et al. CIKM09] L atent d YN amical D irichlet A llocation Tomonari MASADA ISNN 2010
  • 11. Tomonari MASADA ISNN 2010 LYNDA unnormalized Gaussian densities
  • 12.
  • 13.
  • 14. x y z θ α φ s τ η γ β b a
  • 15.
  • 16. Tomonari MASADA ISNN 2010 entire document set ground-truth document set “ Yes!” “ No!”
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22. Tomonari MASADA ISNN 2010 http://www.cis.nagasaki-u.ac.jp/~masada/researches.html DBLP 1990 ~ 2009 LDA BTOT
  • 23. Tomonari MASADA ISNN 2010 http://www.cis.nagasaki-u.ac.jp/~masada/researches.html Xinhua net May 5, 2009 ~ Dec. 17, 2009 LDA BTOT
  • 24. Tomonari MASADA ISNN 2010 http://www.cis.nagasaki-u.ac.jp/~masada/researches.html Yomiuri Newspaper 2002 ~ 2005 LDA BTOT