Fergus, Perona and Zisserman, “Object Class Recognition by Unsupervised Scale-Invariant Learning”, CVPR 03. Presented by Yuval Netzer

1. Constellation Models and Unsupervised Learning for Object Class Recognition Fergus, Perona and Zisserman, “ Object Class Recognition by Unsupervised Scale-Invariant Learning”, CVPR 03. Presented by Yuval Netzer

2. Slides Credit: Gary Bradski, Sebastian Thrun, Rob Fergus, Pietro Perona, Andrew Zisserman, Li Fei-Fei, Antonio Torralba

3. 3 main issues to address: Representation Learning Recognition Difficulties: Size variation Background clutter Occlusion Intra-class variation “ Unsupervised” learning! 10000s of categories.. Task: Recognition of object categories “ Object class recognition using unsupervised scale-invariant learning”

9. Generative vs. Discriminative

11. Foreground model Joint Gaussian shape pdf Pp oisson ( N |  ) p ( x ) = A -1 (uniform) Poission pdf on # detections Uniform shape pdf Prob. of detection table Generative probabilistic model Clutter model Gaussian relative scale pdf Log(scale) Gaussian appearance pdf Uniform relative scale pdf Log(scale) Gaussian part appearance pdf 0.8 0.75 0.9

12. Foreground model Joint Gaussian shape pdf Pp oisson ( N |  ) p ( x ) = A -1 (uniform) Poission pdf on # detections Uniform shape pdf Prob. of detection table Generative probabilistic model Clutter model Gaussian relative scale pdf Log(scale) Gaussian appearance pdf Uniform relative scale pdf Log(scale) Gaussian part appearance pdf 0.8 0.75 0.9

13. Scale, Saliency and Image Description. Timor Kadir and Michael Brady.

15. p(A)=  parts G (A part | c part , V part )

17. Combining it together for Recognition : X – Shape (features locations) A – Appearance (feature representations) S – Scale (vector of feature relative scales) Assume we learned model parameters θ Extract X, S, A from image Make a Bayesian decision: We apply threshold to the likelihood ratio R to decide whether the input image belongs to the class (so priors ratio may be set to 1). Approx. using (delta function) All non object images modeled using a single set of parameters

19. Recognition (cont.) The term p(X, S, A | θ) can be factored into: Each of the terms has a closed (computable) form given the model parameters θ h – Hypothesis (which part is represented by which observed feature)

20. Foreground model Prob. of detection table Gaussian part appearance pdf Gaussian relative scale pdf Log(scale) Can represent both geometrically constrained objects and appearance distinctive objects lacking geometric form Gaussian shape pdf 0.8 0.75 0.9

23. Example scheme, using EM for maximum likelihood learning Given an initial estimate of  : ... Image 1 Image 2 Image i 1. Assign probabilities to constellations Large P Small P 2. Use probabilities as weights to re-estimate parameters. Example:  Large P x + Small P x pdf new estimate of  + … =

25. posterior ratio likelihood ratio prior ratio

27. Motorbikes Pre flipped to face to the right

28. Motorbikes

29. Motorbikes Equal error rate: 7.5% Tight shape model

30. Background images

31. Frontal faces Equal error rate: 4.6%

32. Airplanes Equal error rate: 9.8%

33. Scale-invariant Spotted cats Equal error rate: 10.0% Loose shape model

34. Scale-invariant cars Equal error rate: 9.7%

35. ROC equal error rates Pre-scaled data (identical settings): Scale-invariant learning and recognition:

36. Scale-invariant cars

37. ROC equal error rates Pre-scaled data (identical settings): Scale-invariant learning and recognition:

38. Robustness of algorithm