Constellation Models and Unsupervised Learning for Object Class Recognition

Constellation Models and Unsupervised Learning for Object Class Recognition Fergus, Perona and Zisserman, “ Object Class Recognition by Unsupervised Scale-Invariant Learning”, CVPR 03. Presented by Yuval Netzer

Slides Credit: Gary Bradski, Sebastian Thrun, Rob Fergus, Pietro Perona, Andrew Zisserman, Li Fei-Fei, Antonio Torralba

3 main issues to address: Representation Learning Recognition Difficulties: Size variation Background clutter Occlusion Intra-class variation “ Unsupervised” learning! 10000s of categories.. Task: Recognition of object categories “ Object class recognition using unsupervised scale-invariant learning”

Object as set of parts

Explicitly modeling:

Relative locations and scales between parts

Appearance of each part, Occlusion

How to model location?

How to represent appearance?

Sparse or dense (pixels or regions)?

How to handle occlusion/clutter?

Representation: Constellation of Parts

Object categorization: the statistical viewpoint vs.

Bayes rule:

posterior ratio likelihood ratio prior ratio

Object categorization: the statistical viewpoint posterior ratio likelihood ratio prior ratio

Discriminative methods model posterior

Generative methods model likelihood and prior

Discriminative

Direct modeling of

Zebra Non-zebra Decision boundary

Generative

Model and

Middle  Low High Middle Low

Generative vs. Discriminative

Generative Methods

 Relatively straightforward to characterize invariances

 They wastefully model variability which is unimportant for classification

Discriminative Methods  They use the flexibility of the model in relevant regions of input space  They interpolate between training examples, and hence can fail if novel inputs are presented

Foreground model Joint Gaussian shape pdf Pp oisson ( N |  ) p ( x ) = A -1 (uniform) Poission pdf on # detections Uniform shape pdf Prob. of detection table Generative probabilistic model Clutter model Gaussian relative scale pdf Log(scale) Gaussian appearance pdf Uniform relative scale pdf Log(scale) Gaussian part appearance pdf 0.8 0.75 0.9

Scale, Saliency and Image Description. Timor Kadir and Michael Brady.

p(A)=  parts G (A part | c part , V part )

Sparse representation

+ Computationally tractable (10 5 pixels  10 1 -- 10 2 parts)

+ Generative representation of class

- Throw away most image information

- Parts need to be distinctive to separate from other classes

Combining it together for Recognition : X – Shape (features locations) A – Appearance (feature representations) S – Scale (vector of feature relative scales) Assume we learned model parameters θ Extract X, S, A from image Make a Bayesian decision: We apply threshold to the likelihood ratio R to decide whether the input image belongs to the class (so priors ratio may be set to 1). Approx. using (delta function) All non object images modeled using a single set of parameters

The correspondence problem

Model with P parts

Image with N possible locations for each part

N P combinations!!!

Recognition (cont.) The term p(X, S, A | θ) can be factored into: Each of the terms has a closed (computable) form given the model parameters θ h – Hypothesis (which part is represented by which observed feature)

Foreground model Prob. of detection table Gaussian part appearance pdf Gaussian relative scale pdf Log(scale) Can represent both geometrically constrained objects and appearance distinctive objects lacking geometric form Gaussian shape pdf 0.8 0.75 0.9

Task: Estimation of model parameters

Learning using EM

Let the assignments be a hidden variable and use EM algorithm to learn them and the model parameters

Chicken and Egg type problem, since we initially know neither:

Model parameters

- Assignment of regions to parts

Learning procedure E-step: Compute assignments for which regions belong to which part M-step: Update model parameters

Find regions & their location & appearance

Initialize model parameters

Use EM and iterate to convergence:

Trying to maximize likelihood – consistency in shape & appearance

Example scheme, using EM for maximum likelihood learning Given an initial estimate of  : ... Image 1 Image 2 Image i 1. Assign probabilities to constellations Large P Small P 2. Use probabilities as weights to re-estimate parameters. Example:  Large P x + Small P x pdf new estimate of  + … =

Efficient search methods

Condition on assigned parts to give search regions for remaining ones

A*

posterior ratio likelihood ratio prior ratio

Experimental procedure Two series of experiments: Datasets : Motorbikes, Faces, Spotted cats, Airplanes, Cars from behind and side Between 200 and 800 images in size Training 50% images No identifcation of object within image