1. Object Stereo
2014/3/25 Yichong Bai Object Stereo 1

2. Problem
• Image pair  Disparity map + Object level segmentation
• “A combined algorithm for stereo matching and object
segmentation”
2014/3/25 Yichong Bai Object Stereo 2

3. Overview
• Object level
• Segmentation in object level
• Model in object level
• Pixel level
• Pixel-corresponding needed
• 3D connectivity term
• Works with occlusion
• Slow, 20m/p
2014/3/25 Yichong Bai Object Stereo 3

4. Difference with ours
• Object level  too much semantics
• Initial disparity map needed
• Too slow
2014/3/25 Yichong Bai Object Stereo 4

5. Model
• Scene Representation
• Coarse to fine
• Object = object plane + parallax
• Parallax, surface-based representation
• Object
• Color model
• Parallax model
• Object plane
2014/3/25 Yichong Bai Object Stereo 5

6. Model
• Problem Formulation
• F: Pixel  Planes ( Depth)
• O: Pixel  Object
•
2014/3/25 Yichong Bai Object Stereo 6

7. Model
• Terms
• Photo Consistency
• Pixel dissimilarity
• Occlusion
• Corresponding pixels  same depth plane and object
•
•
2014/3/25 Yichong Bai Object Stereo 7

8. Model
• Terms
• Object Coherence
•
2014/3/25 Yichong Bai Object Stereo 8

9. Model
• Terms
• Depth Plane-Coherency
•
2014/3/25 Yichong Bai Object Stereo 9

10. Model
• Terms
• Object-Color
• GMM
• the probability that a pixel lies inside the object according to its color value
2014/3/25 Yichong Bai Object Stereo 10

11. Model
• Terms
• Object-Parallax
• regularizes the disparities of an object with respect to its object plane
• parallax is likely to be compact
• non-parametric histogram
•
• is histogram probability
2014/3/25 Yichong Bai Object Stereo 11

12. Model
• Terms
• Object-MDL
• Small no. of objects
2014/3/25 Yichong Bai Object Stereo 12

13. Model
• Terms
• 3D Connectivity Prior
• Connectivity = same object OR occluded
• Approximation:
• We randomly sample a large set of pairs of points p and q, where p and q belong
to different connected components of the object. We then draw a line between
p and q and check if all pixels on the line satisfy condition (10).
2014/3/25 Yichong Bai Object Stereo 13

14. Solve
• Energy Minimization
• Fusion move
• Proposal generator
• Like generics algorithm
• Proposal  Fuse  Evaluate
2014/3/25 Yichong Bai Object Stereo 14

15. Solve
• Proposals
• Initialization
• We first compute a disparity map F using the fast stereo matcher
• F is now derived by fitting a plane to each color segment using the initial
disparity map
• To derive O, we take the color segmentation result and group segments that
have similar depths according to F
• GMM: EM
• 30 Initial proposals, with difference segmentations and parameters
2014/3/25 Yichong Bai Object Stereo 15

16. Solve
• Iteration
• Refit
• Fix F, refine O
• Color model, Object plane, Parallax model
• Expansion
• derived by setting all pixels of F to f and all pixels of O to o
2014/3/25 Yichong Bai Object Stereo 16

17. Evaluation
• Middlebury benchmark
• Works best on cone set (artificial planar scene)
2014/3/25 Yichong Bai Object Stereo 17