Spatially coherent latent topic model for concurrent object segmentation and classification<br />Authors: Liangliang Cao, ...
Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
Motivation: Real world problem often full of “noises”<br />Bags of words (local features)<br />Spatial relationships of ob...
Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
Generative vs Discriminative <br />Generative model: model p(x, y) or p(x|y)p(y)<br />Discriminative model: model p(y|x)<b...
Naïve Bayesian model <br />(c: class, w: visual words)<br />Once we have learnt the distribution, for a query image<br />G...
Generative model: Another example<br />Mixture Gaussian Model<br />How to infer from unlabeled data even if we<br />know t...
A graphical model<br />Object class<br />c<br />P(c)<br />Inverse Variance<br />Mean<br />γ<br />μ<br />P(γ|c)<br />P(μ|c)...
Nodes represent variables</li></ul>Hidden<br /><ul><li>Links show dependencies
Conditional distributions   at each node</li></li></ul><li>Inference of latent variables<br />Expectation maximization (EM...
Use Gibbs sampling from the Posterior
Slow to converge
Variational method/Variational Message Passing (VMP)
Algorithms that convert inference problems into optimization problems (Opper and Saad 2001; Wainwright and Jordan 2003)</l...
Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
Back to the topic: the paper<br />bag of words<br />Key Ideas:<br />Latent topics are spatially coherent<br />Generate top...
Homogeneous Appearance ar: average of color or texture features
SIFT-based visual words: wr
Concurrent segmentation and classification</li></li></ul><li>Spatial Latent Topic Model<br />Notation:<br />Image Id<br />...
Spatial Latent Topic Model (Unsupervised)<br />Multinomial<br />Dirichlet<br />prior<br />Maximize Log-likelihood<br />an ...
Variaitional Message Passing (Winn 2005)<br />Coupling hidden variables θ, α, β makes the maximization intractable<br />In...
Variaitional Message Passing (Winn 2005)<br />Further factorization assumptions (Jordan et al., 1999; Jaakkola, 2001; Pari...
Variaitional Message Passing (Winn 2005)<br />Eqn. (6) in the paper<br />Bayesian networks <br />representation<br />Marko...
Spatial Latent Topic Model (Supervised)<br />Now it becomes C x K matrix, i.e. θ depends on observed c<br />For a query im...
Upcoming SlideShare
Loading in …5
×

Spatially Coherent Latent Topic Model For Concurrent Object Segmentation and Classification

2,501 views

Published on

A paper review

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,501
On SlideShare
0
From Embeds
0
Number of Embeds
35
Actions
Shares
0
Downloads
30
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Spatially Coherent Latent Topic Model For Concurrent Object Segmentation and Classification

  1. 1. Spatially coherent latent topic model for concurrent object segmentation and classification<br />Authors: Liangliang Cao, Li Fei-Fei<br />Presenter: Shao-Chuan Wang<br />
  2. 2. Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
  3. 3. Motivation: Real world problem often full of “noises”<br />Bags of words (local features)<br />Spatial relationships of objects are ignored (has its limit)<br />When classify a test image, what is its “subject” ?<br />Flag?<br />Banner?<br />People?<br />Sports field?<br />From Prof. Fei-Fei’s ICCV09 tutorial slide<br />
  4. 4. Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
  5. 5. Generative vs Discriminative <br />Generative model: model p(x, y) or p(x|y)p(y)<br />Discriminative model: model p(y|x)<br />0.1<br />0.05<br />0<br />0<br />10<br />20<br />30<br />40<br />50<br />60<br />70<br />1<br />0.5<br />0<br />0<br />10<br />20<br />30<br />40<br />50<br />60<br />70<br />x = data<br />From Prof. Antonio Torralba course slide<br />
  6. 6. Naïve Bayesian model <br />(c: class, w: visual words)<br />Once we have learnt the distribution, for a query image<br />Generative model: An example<br />Bayesian<br />Networks<br />c<br />w1<br />wn<br />… <br />
  7. 7. Generative model: Another example<br />Mixture Gaussian Model<br />How to infer from unlabeled data even if we<br />know the underlining probability distribution structure? <br />?<br />
  8. 8. A graphical model<br />Object class<br />c<br />P(c)<br />Inverse Variance<br />Mean<br />γ<br />μ<br />P(γ|c)<br />P(μ|c)<br />Observed data<br />x<br />P(x|μ,γ)<br /><ul><li>Directed graph
  9. 9. Nodes represent variables</li></ul>Hidden<br /><ul><li>Links show dependencies
  10. 10. Conditional distributions at each node</li></li></ul><li>Inference of latent variables<br />Expectation maximization (EM)<br />“Soft guess” latent variable first (E-step)<br />Based on latent variable (assume it is correct), solve optimization problem (M-step)<br /><ul><li>Markov-chain Monte Carlo (MCMC)
  11. 11. Use Gibbs sampling from the Posterior
  12. 12. Slow to converge
  13. 13. Variational method/Variational Message Passing (VMP)
  14. 14. Algorithms that convert inference problems into optimization problems (Opper and Saad 2001; Wainwright and Jordan 2003)</li></ul>Image from Wikipedia<br />
  15. 15. Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
  16. 16. Back to the topic: the paper<br />bag of words<br />Key Ideas:<br />Latent topics are spatially coherent<br />Generate topic distribution at the region level<br />Over-segmentation, then merge by same topics<br />Avoid obtaining regions larger than the objects<br />One topic per region<br />Can recognize objects with occlusion<br />oversegmentation<br /><ul><li>Describe a region:
  17. 17. Homogeneous Appearance ar: average of color or texture features
  18. 18. SIFT-based visual words: wr
  19. 19. Concurrent segmentation and classification</li></li></ul><li>Spatial Latent Topic Model<br />Notation:<br />Image Id<br />Region r = {1,2,…,Rd}<br />Latent topic zr= {1,2,…,K}<br />appearance ar = {1,2,…,A}<br />visual words wr = (wr1,wr2,…, wrMr); wr1 = {1,2,…,W}<br />P(zr |θd): <br />topic probability (Multinomial distribution) parameterized by θd<br />P(θd|λ): <br />Dirichlet prior of θd, parameterized by λ<br />α, β: <br />parameters describing the probability of generating appearance and visual words given topic<br />
  20. 20. Spatial Latent Topic Model (Unsupervised)<br />Multinomial<br />Dirichlet<br />prior<br />Maximize Log-likelihood<br />an optimization problem: close-formed solution is intractable <br />
  21. 21. Variaitional Message Passing (Winn 2005)<br />Coupling hidden variables θ, α, β makes the maximization intractable<br />Instead, maximize the lower bound of L <br />Goal: Find a tractable Q(H) that closely approximates the true posterior distribution P(H|V) (equality holds for any distribution Q)<br />←Or equivalently, minimize KL(Q||P)<br />
  22. 22. Variaitional Message Passing (Winn 2005)<br />Further factorization assumptions (Jordan et al., 1999; Jaakkola, 2001; Parisi, 1988) (restrict the family of distributions Q)<br />Entropy term<br />=<br />Where,<br />
  23. 23. Variaitional Message Passing (Winn 2005)<br />Eqn. (6) in the paper<br />Bayesian networks <br />representation<br />Markov blanket:<br />
  24. 24. Spatial Latent Topic Model (Supervised)<br />Now it becomes C x K matrix, i.e. θ depends on observed c<br />For a query image,Id , find its most probable category c: <br />
  25. 25. Process<br />Training step<br />maximize total likelihood of training images, subject λ, α, θ and zr<br />The learned λ, α are fixed<br />Testing phase, for a query Image Id<br />Estimate its θd and zr<br />For classification task, find its most probable latent topics as its category<br />For segmentation task, for the same zr, merge it.<br />(3)<br />
  26. 26. Outline<br />Motivation<br />A Review on Graphical Models<br />Today’s topic: the paper<br />Their Results<br />
  27. 27. Experimental Results<br />Unsupervised segmentation<br />Occlusion case:<br />
  28. 28. Experimental Results<br />Supervised segmentation<br />Dataset<br />13 classes of nature scenes<br /># of training images: 100<br /># of topics: 60<br /># of categories: 13<br />
  29. 29. Experimental Results<br />Supervised classification<br />Dataset<br />28 classes from Caltech 101<br /># of training images: 30<br /># of test images: 30<br /># of topics in category: 28<br /># of topics in clutter: 34<br />6 background classes are <br />left unlabeled<br />
  30. 30. ~ Thank you ~<br />
  31. 31. Variaitional Message Passing<br />Following this framework, and use the graphical model provided by this paper:<br />

×