CVPR2010: Semi-supervised Learning in Vision: Part 2: Theory

880 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
880
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
88
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CVPR2010: Semi-supervised Learning in Vision: Part 2: Theory

  1. 1. Semi-Supervised Learning in Vision Amir Saffari, Christian Leistner, Horst BischofInstitute for Computer Graphics and Vision, Graz University of Technology CVPR San Francisco, June 18, 2010
  2. 2. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedOutline 1 Semi-Supervised Learning 2 Self-Training 3 Generative Models 4 Margin Assumption 5 Cluster and Manifold Assumption 6 Multi-View Learning 7 Large-Scale, Multi-Class SSL, and Online Learning 8 Related Topics Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  3. 3. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedSupervised and Semi-Supervised Learning Supervised learning is all about finding mappings from input (feature) space to output space: f :X →Y Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  4. 4. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedSupervised and Semi-Supervised Learning Supervised learning is all about finding mappings from input (feature) space to output space: f :X →Y Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  5. 5. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedSupervised and Semi-Supervised Learning Supervised learning is all about finding mappings from input (feature) space to output space: f (x; θ ) ∈ F : X → Y Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  6. 6. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedSupervised and Semi-Supervised Learning In Semi-supervised learning we wish to find mappings by using both labeled and unlabeled data: f (x; θ ) ∈ F : X → Y Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  7. 7. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhy Should SSL Make Sense? Learner f (x; θ ) ∈ F : X → Y Labeled data Dl = {(x, y) ∈ X × Y } Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  8. 8. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhy Should SSL Make Sense? Learner f (x; θ ) ∈ F : X → Y Labeled data Dl = {(x, y) ∈ X × Y } Unlabeled data Du = {x ∈ X } Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  9. 9. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhy Should SSL Make Sense? Learner f (x; θ ) ∈ F : X → Y Labeled data Dl = {(x, y) ∈ X × Y } Unlabeled data Du = {x ∈ X } Unsupervised learning: the goal is to recover the structure of the data, eg. clusters, manifolds, low dimensional embeddings, density ... Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  10. 10. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhy Should SSL Make Sense? Learner f (x; θ ) ∈ F : X → Y Labeled data Dl = {(x, y) ∈ X × Y } Unlabeled data Du = {x ∈ X } Unsupervised learning: the goal is to recover the structure of the data, eg. clusters, manifolds, low dimensional embeddings, density ... Density p(x) Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  11. 11. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? Bayes rule: p(y) p(x|y) prior likelihood p(y|x) = p(x) posterior evidence Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  12. 12. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? Bayes rule: p(y) p(x|y) prior likelihood p(y|x) = p(x) posterior evidence MAP decision rule: p(k) p(x|k) y =arg max p(k |x) = arg max ˆ = k k p(x) =arg max p(k) p(x|k) k Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  13. 13. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? When we expect that p(x) (structure of the data) is related to the p(y|x), ie. they share parameters. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  14. 14. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? When we expect that p(x) (structure of the data) is related to the p(y|x), ie. they share parameters. In other words, a better estimation of p(x) can improve the estimation of p(y|x). Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  15. 15. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? When we expect that p(x) (structure of the data) is related to the p(y|x), ie. they share parameters. In other words, a better estimation of p(x) can improve the estimation of p(y|x). Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  16. 16. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? When we expect that p(x) (structure of the data) is related to the p(y|x), ie. they share parameters. In other words, a better estimation of p(x) can improve the estimation of p(y|x). Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  17. 17. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedWhen unlabeled data is going to help? When we expect that p(x) (structure of the data) is related to the p(y|x), ie. they share parameters. In other words, a better estimation of p(x) can improve the estimation of p(y|x). Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  18. 18. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedSemi-Supervised Learning Assumptions Semi-supervised learning often is effective, if the assumptions regarding the relationship between the structure of the data p(x) and the posterior p(y|x) are true for a given problem. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  19. 19. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedComputer Vision Problems: Object Recognition Structure: similar images may contain similar objects. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  20. 20. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedComputer Vision Problems: Object Detection Structure: similar image patches may contain similar objects. Very close patches (over the 2D image neighborhood) may contain the same object. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  21. 21. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedComputer Vision Problems: Object Tracking Structure: similar image patches may contain similar objects. Very close patches (over the 2D image neighborhood and time) may contain the same object. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  22. 22. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedComputer Vision Problems: Object Segmentation Structure: similar pixels may correspond to the same object. Very close pixels (over the 2D image neighborhood and time) may belong to the same object. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  23. 23. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedDifferent Semi-Supervised Learning Settings Semi-supervised classification: f : X → Y , Y = {1, · · · , K }. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  24. 24. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedDifferent Semi-Supervised Learning Settings Semi-supervised classification: f : X → Y , Y = {1, · · · , K }. Semi-supervised regression: f : X → Y , Y = R. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  25. 25. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedDifferent Semi-Supervised Learning Settings Semi-supervised classification: f : X → Y , Y = {1, · · · , K }. Semi-supervised regression: f : X → Y , Y = R. Semi-supervised clustering: constrainted clustering, clustering with pair-wise must-link and cannot-link constraints. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  26. 26. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedTransductive and Inductive SSL Transductive: find f : Dl ∪ Du → Y |Dl ∪Du | . Can not be used for any future example which was not in the training set. Inductive: find f : X → Y . Can be used for any future example, beyond the training set. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  27. 27. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedInteractive Segmentation Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  28. 28. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedOther Uses of Unlabeled Data Unsupervised preprocessing: normalization, standardization, PCA, ICA, ... Graz University of Technology [Torralba ICCV 2009, Mobahi et al. ICML 2009, Ranzato NIPS 2007] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  29. 29. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedOther Uses of Unlabeled Data Unsupervised preprocessing: normalization, standardization, PCA, ICA, ... Feature extraction: bag-of-words Graz University of Technology [Torralba ICCV 2009, Mobahi et al. ICML 2009, Ranzato NIPS 2007] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  30. 30. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedOther Uses of Unlabeled Data Unsupervised preprocessing: normalization, standardization, PCA, ICA, ... Feature extraction: bag-of-words Unsupervised feature learning: deep learning and sparse coding Graz University of Technology [Torralba ICCV 2009, Mobahi et al. ICML 2009, Ranzato NIPS 2007] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  31. 31. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedThe Simplest Approach to SSL Self-training is a meta learning (wrapper) semi-supervised method. Self-Training Inputs: learning algorithm T, labeled set Dl , and unlabeled set Du . For n = 1 to N: 1 Train using the labeled set: f n = T (Dl ). 2 Use f n to classify the unlabeled set: cu = f n (Du ). 3 Create the set of m most confident examples from the unlabeled set: C ⊂ Du . 4 Update the labeled set: Dl ← Dl ∪ {(x, f n (x ))| x ∈ C}. 5 Update the unlabeled set: Du ← Du C . Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  32. 32. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: 1-NN Classifier Graz University of Technology [Zhu TPCL 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  33. 33. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: 1-NN Classifier Graz University of Technology [Zhu TPCL 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  34. 34. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with RF Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  35. 35. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Self-Training with RF (Iteration 5) Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  36. 36. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Self-Training with RF (Iteration 10) Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  37. 37. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedGenerative Models Joint Distribution p(x, y|θ, π ) = p(x|y; θ ) p(y|π ) mixture component πy Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  38. 38. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedGenerative Models Joint Distribution p(x, y|θ, π ) = p(x|y; θ ) p(y|π ) mixture component πy Posterior πy p(x|y; θ ) p(y|x; θ, π ) = ∑k πk p(x|k; θ ) Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  39. 39. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExpectation Maximization Approach Log-likelihood (θ, π; Dl , Du ) = ∑ log πy p(x|y; θ ) +λ ∑ log ∑ πk p(x|k; θ ) (x,y)∈Dl x ∈Du k labeled data unlabeled data Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  40. 40. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExpectation Maximization Approach Log-likelihood (θ, π; Dl , Du ) = ∑ log πy p(x|y; θ ) +λ ∑ log ∑ πk p(x|k; θ ) (x,y)∈Dl x ∈Du k labeled data unlabeled data Expectation Maximization (EM) can be used to estimate the model parameters. EM Inputs: labeled set Dl , and unlabeled set Du , initial model parameters θ0 , π0 . For n = 1 to N: 1 E step: Estimate the posterior p (y | x; θn−1 , πn−1 ) for Du . 2 M step: Update the model parameters, given the posteriors for unlabeled data: θn , πn = arg max (θ, π; Dl , Du ). θ,π Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  41. 41. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Two Gaussians Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  42. 42. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Supervised GMM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  43. 43. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Semi-Supervised EM with GMM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  44. 44. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedAssumptions [Jaakkola et al.] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  45. 45. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedAssumptions [Jaakkola et al.] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  46. 46. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLinear Classifier Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  47. 47. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMaximum Margin Linear Classifier Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  48. 48. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedSVM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  49. 49. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedS3VM and TSVM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  50. 50. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedS3VM SVM with soft margins: binary case y ∈ {−1, 1} without the bias term λ 1 min w 2 + w 2 2 ∑ max(0, 1 − y w, x ) |Dl | (x,y)∈D l margin hinge loss Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  51. 51. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedS3VM SVM with soft margins: binary case y ∈ {−1, 1} without the bias term λ 1 min w 2 + w 2 2 ∑ max(0, 1 − y w, x ) |Dl | (x,y)∈D l margin hinge loss Prediction rule: ˆ y = sign( w, x ) Margin of an unlabeled data: mu (w, x) = | w, x | Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  52. 52. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedS3VM: Loss Functions Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  53. 53. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedS3VM: Formulation S3VM λ 1 min w 2 w 2 2 + ∑ max(0, 1 − y w, x ) + |Dl | (x,y)∈D l margin hinge loss γ + ∑ max(0, 1 − | w, x |) |Du | x∈Du sym. hinge Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  54. 54. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedS3VM: Formulation S3VM λ 1 min w 2 w 2 2 + ∑ max(0, 1 − y w, x ) + |Dl | (x,y)∈D l margin hinge loss γ + ∑ max(0, 1 − | w, x |) |Du | x∈Du sym. hinge Balancing constraint: 1 1 ∑ y = |Du | ∑ w, x | |Dl | (x,y)∈D lx∈Du [Vapnik 1998, Joachims ICML 1999] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  55. 55. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedEntropy Regularization Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  56. 56. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Linear SVM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  57. 57. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Linear TSVM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  58. 58. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Linear TSVM Graz University of Technology [Zhu TPCL 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  59. 59. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Assumption and Margin Maximization Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  60. 60. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Kernels Linear SVM: f (x) = w, x Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  61. 61. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Kernels Linear SVM: f (x) = w, x Nonlinear SVM: f (x ) = w, φ(x ) = ∑ yα x φ(x), φ(x ) (x,y)∈Dl nonlinear mapping K (x,x ) Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  62. 62. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Kernels Linear SVM: f (x) = w, x Nonlinear SVM: f (x ) = w, φ(x ) = ∑ yα x φ(x), φ(x ) (x,y)∈Dl nonlinear mapping K (x,x ) Cluster Kernel: ∑ p I(c(x) == c(x )) Kc (x, x ) = K (x, x ) n [Weston et al. NIPS 2003] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  63. 63. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting Boosting M f (x) = ∑ wm g(x; θm ) m =1 Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  64. 64. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting Boosting M f (x) = ∑ wm g(x; θm ) m =1 Base Learner g(x; θm ) ∈ G : X → Y Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  65. 65. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting Boosting M f (x) = ∑ wm g(x; θm ) m =1 Base Learner g(x; θm ) ∈ G : X → Y Boosting is a linear classifier over the space of base learners G : f (x) = G (x; θ )w Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  66. 66. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting: Learning with Functional Gradient Descent M 1 f (x; β∗ ) = arg min ∑ (x, y; f ) ⇒ f (x; β∗ ) = ∑ wm g(x; θm ) |Xl | (x,y)∈X ∗ ∗ β l m =1 Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  67. 67. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting: Learning with Functional Gradient Descent M 1 f (x; β∗ ) = arg min ∑ (x, y; f ) ⇒ f (x; β∗ ) = ∑ wm g(x; θm ) |Xl | (x,y)∈X ∗ ∗ β l m =1 Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  68. 68. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting with Prior Priors ∀x ∈ Du , k ∈ Y : q(k |x) 1 min w,θ |Dl | ∑ (x, y; w, θ ) + (x,y)∈Dl Supervised γ |Du | x∑ u +  p (x, q; w, θ ) ∈D Prior [Saffari et al. ECCV 2008, CVPR 2009, ECCV 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  69. 69. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Prior Graz University of Technology [Saffari et al. CVPR 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  70. 70. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Prior Graz University of Technology [Saffari et al. CVPR 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  71. 71. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCluster Prior Graz University of Technology [Saffari et al. CVPR 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  72. 72. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: SSL Boosting with Cluster Prior 0.60 VOC2006 0.55 0.50 Class. Acc. 0.45 0.40 0.35 RMSB SER 0.30 AML SVM TSVM 0.25 0.0 0.1 0.2 0.3 0.4 0.5 Labeled Samp. Ratio, r Graz University of Technology [Saffari et al. CVPR 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  73. 73. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: SSL Boosting with Cluster Prior 10000 Computation Time r =0.1 r =0.5 8000 Time (sec) 6000 4000 2000 0 RMSB SER TSVM Graz University of Technology [Saffari et al. CVPR 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  74. 74. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Boosting Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  75. 75. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Cluster Prior Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  76. 76. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedManifold Assumption [Hein and von Luxburg MLSS 2007] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  77. 77. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedManifold Assumption [Hein and von Luxburg MLSS 2007] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  78. 78. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedManifold Assumption [Hein and von Luxburg MLSS 2007] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  79. 79. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLabel Propagation: Supervised [Zhu TPCL 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  80. 80. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLabel Propagation: Semi-Supervised [Zhu TPCL 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  81. 81. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLabel Propagation: Semi-Supervised [Kveton et al. CVPR OLCV 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  82. 82. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLabel Propagation: Semi-Supervised [Kveton et al. CVPR OLCV 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  83. 83. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMincut Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  84. 84. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMincut Mincut energy: min ∑ {y x } x∈D (x ,y )∈D ∑ s(x, x )(yx − y )2 + ∑ s(x, x )(yx − yx )2 u l x ∈Du Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  85. 85. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedRandom Walks on Graph Graz University of Technology [Zhu TPCL 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  86. 86. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedManifold Regularization: LapSVM LapSVM λ 1 min w 2 w 2 2 + ∑ max(0, 1 − y w, x )+ |Dl | (x,y)∈D l γ + |Du |2 ∑ ∑ s(x, x )( w, x − w, x )2 x∈Du x ∈Dl ∪Du Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  87. 87. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedManifold Regularization: LapSVM LapSVM λ 1 min w 2 w 2 2 + ∑ max(0, 1 − y w, x )+ |Dl | (x,y)∈D l γ + |Du |2 ∑ ∑ s(x, x )( w, x − w, x )2 x∈Du x ∈Dl ∪Du Graph Laplacian λ 1 min w 2 w 2 2 + ∑ max(0, 1 − y w, x )+ |Dl | (x,y)∈D l γ + f, Lf |Du |2 where f is the response vector of the classifier to both labeled and unlabeled data. Graz University of Technology [Belkin et al. JMLR 2006] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  88. 88. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting with Priors and Manifolds 1 min w,θ ∑ (x, y; w, θ ) + |Dl | (x,y)∈D l Supervised γ s(x, x ) + ∑ λ  p (x, q; w, θ ) +(1 − λ) ∑ z(x) m (x, x ; w, θ ) |Du | x∈Du x ∈Du Prior x =x Manifold Graz University of Technology Unsupervised Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  89. 89. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Boosting with Cluster and Manifold Priors Methods-Dataset g241c g241d Digit1 USPS COIL BCI 1-NN 44.05 43.22 23.47 19.82 65.91 48.74 SVM 47.32 46.66 30.60 20.03 68.36 49.85 RF (weak) 47.51 48.44 42.42 22.90 75.72 49.35 GBoost-RF 46.77 46.61 38.70 20.89 69.85 49.12 TSVM 24.71 50.08 17.77 25.20 67.50 49.15 Cluster Kernel 48.28 42.05 18.73 19.41 67.32 48.31 LapSVM 46.21 45.15 08.97 19.05 N/A 49.25 ManifoldBoost 42.17 42.80 19.42 19.97 N/A 47.12 SemiBoost-RF 48.41 47.19 10.57 15.83 63.39 49.77 MCSSB-RF 49.77 48.57 38.50 22.95 69.96 49.12 GPMBoost-RF 15.69 39.45 11.19 14.92 62.60 49.27 [Saffari et al. 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  90. 90. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Boosting with Cluster and Manifold Priors Methods-Dataset g241c g241d Digit1 USPS COIL BCI 1-NN 40.28 37.49 06.12 07.64 23.27 44.83 SVM 23.11 24.64 05.53 09.75 22.93 34.31 RF (weak) 44.23 45.02 17.80 16.73 34.26 43.78 GBoost-RF 31.84 32.38 06.24 13.81 21.88 40.08 TSVM 18.46 22.42 06.15 09.77 25.80 33.25 Cluster Kernel 13.49 04.95 03.79 09.68 21.99 35.17 LapSVM 23.82 26.36 03.13 04.70 N/A 32.39 ManifoldBoost 22.87 25.00 04.29 06.65 N/A 32.17 SemiBoost-RF 41.26 39.14 02.56 05.92 15.31 47.12 MCSSB-RF 45.11 40.26 13.19 12.31 31.09 47.64 GPMBoost-RF 12.80 12.59 02.35 06.33 14.49 45.41 [Saffari et al. 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  91. 91. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Cluster Prior Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  92. 92. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Cluster Prior and Manifolds Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  93. 93. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLearning from Different Views Data is represented by multiple views: x = [x1 | · · · |xV ] T T T Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  94. 94. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLearning from Different Views Data is represented by multiple views: x = [x1 | · · · |xV ] T T T There is a classifier per view: F = { f v }V=1 v Multi-View Learning F ∗ = arg min ∑ (x, y; F ) + γ ∑ φ(x; F ) F (x,y)∈Dl x∈Du Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  95. 95. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training Co-Training Inputs: learning algorithm T, labeled set Dl , and unlabeled set Du . For n = 1 to N: 1 1 2 1 Train using the labeled set: f n = T (Dl ) and f n = T (Dl ).2 2 Use f n1 and f 2 to classify the unlabeled set: c1 = f 1 (D 1 ) and n u n u c2 = f n (Du ). u 2 2 3 Create the set of m most confident examples from each view of the unlabeled set: C 1 ⊂ Du andC 2 ⊂ Du . 1 2 4 Update the labeled set: Dl ← Dl ∪ {(x, f n (x))| x ∈ C 1 } ∪ {(x, f n (x))| x ∈ C 2 }. 1 2 5 Update the unlabeled set: Du ← Du (C 1 ∪ C 2 ). [Blum and Mitchell COLT 1998] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  96. 96. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training and Assumptions We can represent the data in two views: x = [x1 , x2 ]. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  97. 97. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training and Assumptions We can represent the data in two views: x = [x1 , x2 ]. We can train a good classifier only using either x1 or x2 . [Blum and Mitchell COLT 1998, Sindhwani et al. ICML 2005] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  98. 98. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training and Assumptions We can represent the data in two views: x = [x1 , x2 ]. We can train a good classifier only using either x1 or x2 . x1 and x2 are conditionally independent given the class. [Blum and Mitchell COLT 1998, Sindhwani et al. ICML 2005] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  99. 99. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training and Assumptions We can represent the data in two views: x = [x1 , x2 ]. We can train a good classifier only using either x1 or x2 . x1 and x2 are conditionally independent given the class. [Blum and Mitchell COLT 1998, Sindhwani et al. ICML 2005] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  100. 100. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training: Visual and EEG Data Graz University of Technology [Kapoor et al. CVPR 2008] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  101. 101. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training: Visual and EEG Data [Kapoor et al. CVPR 2008] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  102. 102. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedCo-Training: Visual and EEG Data Graz University of Technology [Kapoor et al. CVPR 2008] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  103. 103. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-View Boosting with Priors Multi-View Priors 1 V − 1 s∑ qv (k |xv ) = ps (k |xs ), ∀v ∈ {1, · · · , V }, k ∈ Y =v Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  104. 104. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-View Boosting with Priors Multi-View Priors 1 V − 1 s∑ qv (k |xv ) = ps (k |xs ), ∀v ∈ {1, · · · , V }, k ∈ Y =v Boosting with Priors: 1 arg min ∑ (x, y; w, θ ) + |Dl | (x,y)∈D w,θ l Supervised γ + ∑  p (x, q; w, θ ) |Du | x∈Du Prior [Saffari et al. ECCV 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  105. 105. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedRobust Loss Functions q + =0.5 5 5 0-1 KL Hinge SKL 4 Exponential 4 JS Logit Savage 3 3 (x,y;f) (x,q;f) 2 2 1 1 0 4 2 0 2 4 0 4 2 0 2 4 m(x,y;f) f + (x) q + =0.75 5 KL SKL 4 JS 3 (x,q;f) 2 1 0 4 2 0 2 4 f + (x) Graz University of Technology [Saffari et al. ECCV 2010] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  106. 106. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with RF Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  107. 107. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Linear SVM Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  108. 108. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Multi-ViewClassifiers Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  109. 109. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedExample: Interactive Segmentation with Multi-ViewClassifiers Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  110. 110. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLarge-Scale Applications Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  111. 111. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLarge-Scale Applications Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  112. 112. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLarge-Scale Applications Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  113. 113. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLearning from Gigantic Image Collections Original Objective J (f) = (f − y)T A(f − y) + f T Lf Objective with eigenvectors using f = Uα J (α) = (Uα − y)T A(Uα − y) + α T Σα T Graz University of Technology [Fergus et al. NIPS 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  114. 114. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLearning from Gigantic Image Collections 80 million tiny images [Torralba PAMI 2008, Fergus et al. NIPS 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  115. 115. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLearning from Gigantic Image Collections Graz University of Technology [Fergus et al. NIPS 2009] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  116. 116. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedDeep Learning Embedding loss ∑ ( f ( x ), y ) + γ ∑ ∑ s(x, x ) ( f (x), f (x )) (x,y)∈Dl x∈Du x ∈D Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  117. 117. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedDeep Learning Semantic Role Labeling: “The cat eats the fish in the pond” → “TheARG0 catARG0 eatsREL theARG1 fishARG1 inARGM-LOC theARGM-LOC pondARGM-LOC ” Trained with stochastic gradient descent on 1 million labeled data and 631 million unlabeled data from Wikipedia. Graz University of Technology [Weston et al. ICML 2008] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  118. 118. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedOnline Learning and SSL More about online semi-supervised learning in the second part of the tutorial. [Goldberg et al. ECML 2009, Zeisl et al. CVPR 2010, Leistner et al. PR 2010, Saffari et al. ECCV 2010, Kveton CVPR OLCV 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  119. 119. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-Class Problems Many interesting problems are multi-class. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  120. 120. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-Class Problems What is wrong with 1-vs-all? Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  121. 121. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-Class Problems What is wrong with 1-vs-all? Computational problems. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  122. 122. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-Class Problems What is wrong with 1-vs-all? Computational problems. Calibration problems [B. Schoelkopf, A. Smola, 2002]. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  123. 123. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMulti-Class Problems What is wrong with 1-vs-all? Computational problems. Calibration problems [B. Schoelkopf, A. Smola, 2002]. Artificial unbalanced binary problems. Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  124. 124. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedBoosting with Priors and Manifolds Boosting with Priors and Manifolds is an inherently multi-class algorithm. Graz University of Technology [Saffari et al. CVPR 2009, CVPR 2010, ECCV 2010] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  125. 125. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedTransfer Learning [Pan and Yang TKDE 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  126. 126. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedTransfer Learning [Pan and Yang TKDE 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  127. 127. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedTransfer Learning with Attributes [Farhadi CVPR 2009, Lampert CVPR 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  128. 128. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedTransfer Learning with Attributes Graz University of Technology [Farhadi et al. CVPR 2010] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  129. 129. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedTransfer Learning with Attributes [Rohbach et al. CVPR 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  130. 130. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLearning from Latent (Hidden) Variables Latent SVM f w (x) = max w, φ(x, z) z λ 1 min w 2 w 2 2 + ∑ max(0, 1 − y f w (x)) |Dl | (x,y)∈D l Graz University of Technology [Felzenszwalb et al. CVPR 2008] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  131. 131. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedIndoor Scene Labeling [Wang et al. ECCV 2010] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  132. 132. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLatent Structural SVM [Yu and Joachims 2009] Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  133. 133. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedLatent Structural SVM for Indoor Scene Labeling Graz University of Technology [Wang et al. ECCV 2010] Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision
  134. 134. SSL Self-Training Generative Models Margin Manifold Multi-View Large-Scale RelatedMultiple Instance Learning - - - - + - - + - - - - + - Graz University of Technology Amir Saffari, Christian Leistner, Horst Bischof Semi-Supervised Learning in Vision

×