• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
[Kim+ ICML2012] Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data
 

[Kim+ ICML2012] Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data

on

  • 17,259 views

Supervised Nonparametric Topic Model

Supervised Nonparametric Topic Model

Statistics

Views

Total Views
17,259
Views on SlideShare
2,019
Embed Views
15,240

Actions

Likes
0
Downloads
38
Comments
0

16 Embeds 15,240

http://shuyo.wordpress.com 8643
http://d.hatena.ne.jp 6360
http://hatenatunnel.appspot.com 160
http://webcache.googleusercontent.com 25
http://www.hatenatunnel.appspot.com 11
http://cloud.feedly.com 9
https://si0.twimg.com 7
http://translate.googleusercontent.com 5
http://prlog.ru 4
https://shuyo.wordpress.com 4
http://digg.com 3
https://twimg0-a.akamaihd.net 3
http://www.365dailyjournal.com 2
http://feedly.com 2
http://www.google.com 1
https://twitter.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    [Kim+ ICML2012] Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data [Kim+ ICML2012] Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data Presentation Transcript

    • [Kim+ ICML2012] Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data 2012/07/28 Nakatani Shuyo @ Cybozu Labs, Inc twitter : @shuyo
    • LDA(Latent Dirichlet Allocation) [Blei+ 03]• Unsupervised Topic Model – Each word has an unobserved topic• Parametric – The topic size K is given in advance via Wikipedia
    • Labeled LDA [Ramage+ 09]• Supervised Topic Model – Each document has an observed label• Parametric via [Ramage+ 09]
    • Generative Process for L-LDA• 𝜷 𝑘 ~Dir 𝜼 topics corresponding to 𝑑 observed labels• Λ 𝑘 ~Bernoulli Φ 𝑘• 𝜽 𝑑 ~Dir 𝜶 𝑑 restricted to labeled – where 𝜶 𝑑 = 𝛼𝑘 parameters 𝑑 𝑘 Λ 𝑘 =1 𝑑 𝑑• 𝑧 𝑖 ~Multi 𝜽 𝑑• 𝑤𝑖 ~Multi 𝜷 𝑧 𝑑 𝑖 via [Ramage+ 09]
    • Pros/Cons of L-LDA• Pros – Easy to implement• Cons via [Ramage+ 09] – It is necessary to specify label-topic correspondence manually • Its performance depends on the corresponds ※) My implementation is here : https://github.com/shuyo/iir/blob/master/lda/llda.py
    • DP-MRM [Kim+ 12] – Dirichlet Process with Mixed Random Measures• Supervised Topic Model• Nonparametric – K is not the topic size, but the label size 𝛼 𝑁𝑗 𝐻 𝐺0𝑘 𝐺𝑗 𝜃 𝑗𝑖 𝑥 𝑗𝑖 𝜆j 𝑟𝑗 𝐷 𝛽 𝛾𝑘 𝜂 𝐾
    • Generative Process for DP-MRM 𝛼 Each label has a random measure as topic space 𝑁𝑗 𝐻 𝐺0𝑘 𝐺𝑗 𝜃 𝑗𝑖 𝑥 𝑗𝑖• 𝐻 = Dir 𝛽 𝜆j 𝑟𝑗 𝐷• 𝐺0𝑘 ~DP 𝛾 𝑘 , 𝐻 𝛽 𝐾 𝛾𝑘 𝜂• 𝜆 𝑗 ~Dir 𝒓 𝑗 𝜂 where 𝒓 𝑗 = 𝐼 𝑘∈label 𝑗• 𝐺 𝑗 ~DP 𝛼, 𝑘∈label 𝑗 𝜆 𝑗𝑘 𝐺0𝑘 mixed random measures• 𝜃 𝑗𝑖 ~𝐺 𝑗 , 𝑥 𝑗𝑖 ~𝐹 𝜃 𝑗𝑖 = Multi 𝜃 𝑗𝑖
    • Stick Breaking Process• 𝑣 𝑙 𝑘 ~Beta 1, 𝛾 𝑘 , 𝜋 𝑙𝑘 = 𝑣 𝑙 𝑘 𝑙−1 𝑑=0 1 − 𝑣 𝑑𝑘• 𝜙 𝑙𝑘 ~𝐻, 𝐺0𝑘 = ∞ 𝑙=0 𝜋 𝑙𝑘 𝛿 𝜙 𝑘 𝑙 𝑡−1• 𝜆 𝑗 ~Dir 𝒓 𝑗 𝜂 , 𝑤 𝑗𝑡 ~Beta 1, 𝛼 , 𝜋 𝑗𝑡 = 𝑤 𝑗𝑡 𝑑=0 1 − 𝑤 𝑗𝑑 𝑘 𝑗𝑡 ∞• 𝑘 𝑗𝑡 ~Multi 𝜆 𝑗 , 𝜓 𝑗𝑡 ~𝐺0 , 𝐺𝑗 = 𝑡=0 𝜋 𝑗𝑡 𝛿 𝜓 𝑗𝑡
    • Chinese Restaurant Franchise• 𝑡 𝑗𝑖 : table index of 𝑖-th term in 𝑗-th document• 𝑘 𝑗𝑡 , 𝑙 𝑗𝑡 : dish indexes on 𝑡-th table of 𝑗-th document This layer consists on only a single DP G0 on normal HDP
    • Inference (1)• Sampling 𝑡
    • Inference (2)• Sampling 𝑘 and 𝑙
    • Experiments• DP-MRM gives label-topic probabilistic corresponding automatically. via [Kim+ 12]
    • via [Kim+ 12]• L-LDA can also predict single labeled document to assign a common second label to any documents.
    • References• [Kim+ ICML2012] Dirichlet Process with Mixed Random Measures : A Nonparametric Topic Model for Labeled Data• [Ramage+ EMNLP2009] Labeled LDA : A supervised topic model for credit attribution in multi-labeled corpora• [Blei+ 2003] Latent Dirichlet Allocation