• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Cvpr2007 object category recognition   p4 - combined segmentation and recognition
 

Cvpr2007 object category recognition p4 - combined segmentation and recognition

on

  • 529 views

 

Statistics

Views

Total Views
529
Views on SlideShare
529
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Different occlusions preserves ordering, deformations preserve ordering
  • Different occlusions preserves ordering, deformations preserve ordering
  • Edge weight larger at image edges
  • Write down the contribution part of this paper
  • Emphasise class model (shared) – all other variables per-image. Emphasise LEARN EVERYTHING SIMULTANEOUSLY.

Cvpr2007 object category recognition   p4 - combined segmentation and recognition Cvpr2007 object category recognition p4 - combined segmentation and recognition Presentation Transcript

  • Part 4: Combined segmentation and recognition by Rob Fergus (MIT)
  • Aim
    • Given an image and object category, to segment the object
    • Segmentation should (ideally) be
    • shaped like the object e.g. cow-like
    • obtained efficiently in an unsupervised manner
    • able to handle self-occlusion
    Segmentation Object Category Model Cow Image Segmented Cow Slide from Kumar ‘05
  • Feature-detector view
  •  
  •  
  •  
  • Examples of bottom-up segmentation
    • Using Normalized Cuts, Shi & Malik, 1997
    Borenstein and Ullman, ECCV 2002
  • Jigsaw approach: Borenstein and Ullman, 2002
  • Implicit Shape Model - Liebe and Schiele, 2003 Liebe and Schiele, 2003, 2005 Backprojected Hypotheses Interest Points Matched Codebook Entries Probabilistic Voting Voting Space (continuous) Backprojection of Maxima Segmentation Refined Hypotheses (uniform sampling)
  • Random Fields for segmentation I = Image pixels (observed) h = foreground/background labels (hidden) – one label per pixel  = Parameters Prior Likelihood Posterior Joint
    • Generative approach models joint
      •  Markov random field (MRF)
    • 2. Discriminative approach models posterior directly
    •  Conditional random field (CRF)
  • Generative Markov Random Field I (pixels) Image Plane i j Prior has no dependency on I h (labels)  {foreground,background} h i h j Unary Potential  i ( I |h i ,  i ) Pairwise Potential (MRF)  ij (h i , h j |  ij ) MRF Prior Likelihood
  • Conditional Random Field Lafferty, McCallum and Pereira 2001 Pairwise Unary
    • Dependency on I allows introduction of pairwise terms that make use of image.
    • For example, neighboring labels should be similar only if pixel colors are similar  Contrast term
    Discriminative approach e.g Kumar and Hebert 2003 I (pixels) Image Plane i j h i h j
  • OBJCUT Ω (shape parameter) Kumar, Torr & Zisserman 2005 Pairwise Unary
    • Ω is a shape prior on the labels from a Layered Pictorial Structure (LPS) model
    • Segmentation by:
      • - Match LPS model to image (get number of samples, each with a different pose
      • Marginalize over the samples using a single graph cut
      • [Boykov & Jolly, 2001]
    Label smoothness Contrast Distance from Ω Color Likelihood I (pixels) Image Plane i j h i h j Figure from Kumar et al., CVPR 2005
  • OBJCUT: Shape prior - Ω - Layered Pictorial Structures (LPS)
    • Generative model
    • Composition of parts + spatial layout
    Layer 2 Layer 1 Parts in Layer 2 can occlude parts in Layer 1 Spatial Layout (Pairwise Configuration) Kumar, et al. 2004, 2005
  • OBJCUT: Results In the absence of a clear boundary between object and background Segmentation Image Using LPS Model for Cow
  • Levin & Weiss [ECCV 2006] Segmentation alignment with image edges Consistency with fragments segmentation Resulting min-cut segmentation
  • Winn and Shotton 2006 Layout Consistent Random Field [Lepetit et al. CVPR 2005]
    • Decision forest classifier
    • Features are differences of pixel intensities
    Classifier
  • Layout consistency Neighboring pixels (p,q) ? (p,q+1) (p,q) (p+1,q+1) (p-1,q+1) Layout consistent Winn and Shotton 2006 (8,3) (9,3) (7,3) (8,2) (9,2) (7,2) (8,4) (9,4) (7,4)
  • Layout Consistent Random Field Winn and Shotton 2006 Layout consistency Part detector
  • Stability of part labelling Part color key
  • Object-Specific Figure-Ground Segregation Stella X. Yu and Jianbo Shi, 2002
  • Image parsing: Tu, Zhu and Yuille 2003
  • Image parsing: Tu, Zhu and Yuille 2003
  • Segment out all the cars … . fused tree model for cars Unseen image Training images Segmented Cars Segmentation Trees Overview Multiscale Seg. Todorovic and Ahuja, CVPR 2006 Slide from T. Wu
  • LOCUS model Deformation field D Position & size T Class shape π Class edge sprite μ o , σ o Edge image e Image Object appearance λ 1 Background appearance λ 0 Mask m Shared between images Different for each image Kannan, Jojic and Frey 2004 Winn and Jojic, 2005
  • In this section: brief paper reviews
    • Jigsaw approach: Borenstein & Ullman, 2001, 2002
    • Concurrent recognition and segmentation: Yu and Shi, 2002
    • Image parsing: Tu, Zhu & Yuille 2003
    • Interleaved segmentation: Liebe & Schiele, 2004, 2005
    • OBJCUT: Kumar, Torr, Zisserman 2005
    • LOCUS: Winn and Jojic, 2005
    • LayoutCRF: Winn and Shotton, 2006
    • Levin and Weiss, 2006
    • Todorovic and Ahuja, 2006
  • Summary
    • Strength
      • Explains every pixel of the image
      • Useful for image editing, layering, etc.
    • Issues
      • Invariance issues
        • (especially) scale, view-point variations
      • Inference difficulties
  •  
  •  
  • Conditional Random Fields for Segmentation
    • Segmentation map x
    • Image I
    Low-level pairwise term High-level local term Pixel-wise similarity
  • Object-Specific Figure-Ground Segregation Some segmentation/detection results Yu and Shi, 2002
    • Multiscale Conditional Random Fields for Image Labeling
    • Xuming He Richard S. Zemel Miguel A´ . Carreira-Perpin˜a´n
    • Conditional Random Fields for Object
    • Recognition
    • Ariadna Quattoni Michael Collins Trevor Darrell
  • OBJCUT
    • Probability of labelling in addition has
    • Unary potential which depend on distance from Θ (shape parameter)
    D (pixels) m (labels) Θ (shape parameter) Image Plane Object Category Specific MRF x y m x m y Unary Potential Φ x (m x | Θ ) Kumar, et al. 2004, 2005
  • Localization using features
  • Levin and Weiss 2006 Levin and Weiss, ECCV 2006
  • Results: horses
  • Results: horses
  • Cows: Results
    • Segmentations from interest points
      • Single-frame recognition - No temporal continuity used!
    Liebe and Schiele, 2003, 2005
  •  
  • Examples of low-level image segmentation
    • Normalized Cuts, Shi & Malik, 1997
    Borenstein & Ullman, ECCV 2002
  •  
  • Jigsaw approach
    • Each patch has foreground/background mask
  • LayoutCRF
  •  
  • Segmentation
    • Interpretation of p(figure) map
      • per-pixel confidence in object hypothesis
      • Use for hypothesis verification
    Liebe and Schiele, 2003, 2005 p(figure) p(ground) Segmentation p(figure) p(ground) Original image