• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Disparity Estimation Using A Color Segmentation V3
 

Disparity Estimation Using A Color Segmentation V3

on

  • 1,435 views

Master_Project

Master_Project

Statistics

Views

Total Views
1,435
Views on SlideShare
1,433
Embed Views
2

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 2

http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Disparity Estimation Using A Color Segmentation V3 Disparity Estimation Using A Color Segmentation V3 Presentation Transcript

    • Disparity Estimation using a Color Segmentation Xin Wang Barcelona, 3rd Sep. 2009
    • Outline for the presentation
      • Task and Problem Identification
      • Algorithmic Overview
      • Segmentation-based Stereo Matching
        • Local Matching
        • Segmentation-based Disparity Fitting
        • Disparity Refinement via Region Merging
      • Experiment Result and Analysis
      • Conclusions and Future Work
      • Questions
    • Task and Problem Identification
      • Stereo Correspondence Problem
      • Given an element in the left image, we search for the corresponding element
      • in the right image.
        • Disparity is the displacement for pixel positions between two corresponding points in these images.
        • Epipolar Rectification allows simplifying the search problem from
        • 2D to 1D.
    • Task and Problem Identification
      • Challenges
        • Textureless Region : Matching in an untextured area becomes ambiguous, since there are many potential matching points of very similar intensity patterns.
        • Depth Discontinuity Region : Pixels whose neighbouring disparities differ by more than a given value, usually appear at the boundaries of different scene objects.
        • Occluded Region : Zones that are occluded in the matching image.
    • Task and Problem Identification
      • Quality Measure :
        • Reconstruction :
        • To reconstruct the reference image in a back-ward way, is using
        • reference image’s disparity map and the other view image for
        • reconstruction.
        • Reconstruction Image PSNR (Peak Signal to Noise Ration) :
        • , where
      L (ori) R (ori) Disparity (Left) Disparity (Right)
    • Task and Problem Identification
        • Occlusion mask is made from performing L-R check to groundtruth.
        • PSNR is Calculated in non-occluded region.
    • Outline for the presentation
      • Task and Problem Identification
      • Algorithmic Overview
      • Segmentation-based Stereo Matching
        • Local Matching
        • Segmentation-based Disparity Fitting
        • Disparity Refinement via Region Merging
      • Experiment Result and Analysis
      • Conclusions and Future Work
      • Questions
    • Algorithmic Overview
      • Block Diagram of Algorithm
      Initial Disparity Estimation Color Segmentation to extract the label Polynomial Model based Disparity Representation in Each Color Segment (use label information) Disparity Improvement By Region Merging Iterations Final Disparity Map
    • Outline for the presentation
      • Task and Problem Identification
      • Algorithmic Overview
      • Segmentation-based Stereo Matching
        • Local Matching
        • Segmentation-based Disparity Fitting
        • Disparity Refinement via Region Merging
      • Experiment Result and Analysis
      • Conclusions and Future Work
      • Questions
    • Initial Disparity Estimation
      • Window-based Local Matching :
        • Matching Score : is a quantitative similarity measurement for two different
      • pixels. It is the criterion for the solution of correspondence problem.
        • Winner-take-all : disparity associated with the minimum cost value is
        • selected at each pixel.
    • Outline for the presentation
      • Task and Problem Identification
      • Algorithmic Overview
      • Segmentation-based Stereo Matching
        • Local Matching
        • Segmentation-based Disparity Fitting
        • Disparity Refinement via Region Merging
      • Experiment Result and Analysis
      • Conclusions and Future Work
      • Questions
    • Segmentation-based Approach
      • Approach: Segmentation-based approaches divide one or sometimes both images into non-overlapping regions of homogeneous color.
      • Assumptions:
        • 1, Inside a segment of homogeneous colour the disparity values are expected to follow some particular smooth disparity model (constant disparity, planar model, etc)
        • 2, Disparity discontinuities are assumed to coincide with the boundaries
        • of those color regions.
    • Segmentation-based Approach
      • Polynomial Models for Each Segment
        • Order 0 Polynomial Model :
        • Order 1 Polynomial Model :
        • Order 2 Polynomial Model :
      (planar parallel to the camera front plane) (planar model) (parabolic curved surface)
    • Segmentation-based Approach
      • Model Parameters Estimation
        • We use initial disparity as input data
        • Unknown Parameters:
          • Order 0 : 1 parameter - > 1 point
          • Order 1 : 3 parameters -> 3 points
          • Order 2 : 6 parameters -> 6 points
        • Proposed Approaches for Model Fitting:
          • Least Squares
          • Random Sample and Consensus (RANSAC)
    • Segmentation-based Approach
      • Least Squares
        • Over-determined Problem : The number of points in each region is much higher than the number of equations needed to compute the analytical solution.
        • Best Fit for Observed Data : Sum of squared residuals has its least value. (but sensitive to outliers)
        • min( )
    • Segmentation-based Approach
      • RANSAC
      • Non-deterministic algorithm in the sense that it produces a
      • reasonable result only with a certain probability.
        • Assumption : A set of data is that the data consists of inliers and the data's contribution follows a certain set of model parameters, while the outliers do not fit the model.
        • Algorithm: Hypothesize-and-Test framework
          • HYPOTHESIZE: Minimal Sample Set (MSS) is randomly selected from initial input data.
          • TEST: Check the entire dataset with the estimated model from MSS. Data are consistent with the model are called consensus set (CS).
    • Segmentation-based Approach
      • RANSAC in Polynomial Model Fitting
        • Polynomial Order 0 : Median filter
        • Polynomial Order 1 and 2:
          • 1. Select MSS : Randomly select 3 or 6 points from initial disparity points
          • 2. Fit the Model by Solving Linear Equations System:
          • 3. Select Inliers to Form the CS:
      • Distance calculation -> D = d i – f (x i ,y i ,P) if below the distance
      • threshold
          • 4. Choose the best CS to Output Fitted Model: CS with most inliers
    • Segmentation-based Approach
      • Comparison between RANSAC & LS
        • LS : Analytical solution, non-iterative solution. Less computation cost.
        • Very sensitive to outliers.
        • RANSAC: Robust to outliers, non-deterministic algorithm. Accuracy
        • will increase with increasing the iterations and strictness
        • in thresholds.
      • More computation cost and the model may subject to
      • noise.
        • PSNR Comparison Test:
      (Teddy 559 regions, Cones 553 regions, Venus 398 regions)
    • Outline for the presentation
      • Task and Problem Identification
      • Algorithmic Overview
      • Segmentation-based Stereo Matching
        • Local Matching
        • Segmentation-based Disparity Fitting
        • Disparity Refinement via Region Merging
      • Experiment Result and Analysis
      • Conclusions and Future Work
      • Questions
    • Disparity Refinement via Merging
      • Problems : In some regions, due to unproper-segmentation,
      • occlusion, noise, lack of enough inliers, may result in
      • incorrect model estimation.
      • Solution : Merge adjacent “good regions” to re-fit the model.
        • For lack of inliers region : enlarge the area , includes more inliers
        • For occlusion area : get the support from neighbouring good region
        • For small region : large distance separated points, more easy to fit a
        • correct model
    • Disparity Refinement via Merging
      • Merging Block Diagram :
    • Disparity Refinement via Merging
      • Region Similarity Measure Criteria
        • Similarity in Model’s Parameters
        • Number of inliers
        • PSNR of Reconstructed Region
      (Euclidean Distance Based)
    • Disparity Refinement via Merging
      • Implementation for similarity measure:
        • Distance similarity in region’s parameters + PSNR criterion
        • Distance similarity matrix will be calculated:
        • Due to distance similarity based on region’s parameters can not always be true, so I use PSNR criterion for actual merging step to secure merging process will always improve the disparity quality
    • Outline for the presentation
      • Task and Problem Identification
      • Algorithmic Overview
      • Segmentation-based Stereo Matching
        • Local Matching
        • Segmentation-based Disparity Fitting
        • Disparity Refinement via Region Merging
      • Experiment Result and Analysis
      • Conclusions and Future Work
      • Questions
    • Experiment Results
      • Polynomial Model Fitting
        • Compare with initial disparity, usually no improvement
        • Analysis : RANSAC works not well, due to initial color segmentation fails to meet the two assumptions, and some regions lack of enough inliers.
      Parameters : Teddy (559 regions) , Cones (553 regions), Venus (398 regions) RANSAC iterations:200, inlier threshold:0.99 , distance threshold:0.5
    • Experiment Results
      • Mixed Order Polynomial Model Fitting
      Program runs on a 3GHz processor
    • Experiment Results
      • Mixed Order Polynomial Model Fitting
      Red: Order 0, Green: Order 1, Blue: Order 2, Black: No improvement
    • Experiment Results
      • Polynomial order 2 Model Region Merging
      Teddy (559 regions) , Cones (553 regions), Venus (398 regions)
    • Experiment Results
      • Polynomial order 2 Teddy case
    • Experiment Results
      • Polynomial order 1 model results:
      Reconstruction using initial Disparity Reconstruction using merged disparity, 160 rounds
    • Conclusions and Future Work
      • Conclusions:
        • LS is more sensitive to outliers, RANSAC is more robust
        • Sometimes RANSAC will fail to estimate correct parameters, due to the segmented region’s size, noise etc.
        • Small regions may not have enough inliers, region merging could help
        • Bad estimated regions could get support from neighbouring regions in merging process
      • Future work
        • More accurate and computationally efficient region similarity measure should be further studied
        • Try to do merging for all the possibilities for adjacent regions.
        • Convert Matlab code to C++ version
    • Acknowledgement
      • Prof. Josep Ramon Morros i Rubio
      • Albert (Technical support)
      • All Professors and friends in TSC who
      • supported me before.
    • Questions
      • Thank you very much !
      • Questions?
    • Experiment Results
      • Example of Region Merging Improvement (Teddy, order2)
      A, improvement point B, how label changes C, disparity refinement
    • Experiment Results
      • Drop point in the evolution curve (214 & 216region)
    • Experiment Results
      • Order 1 Merging ->
      • Teddy
    • Experiment Results
      • Poor Modeled Region in order2 Teddy
    • Disparity & Depth
      • Disparity is proportional inversely to the depth
    • Experiment Results
      • Local Matching
        • How the local window size affect local matching
        • Too small window can not catch enough intensity variation to give the correct disparity in less-textured regions, too large window can not capture well small details
      Teddy Image Cones Image Venus Image
    • Matching Score
      • Matching Score: Matching measures such as SSD and SAD are strictly assuming the constant color con-straint. So if the computed area fails to meet this constraint, SSD and SAD measures may not be robust. While the other matching scores like gradient-based measure is more robust to changes in camera gain. So we could expect to combine the sum of absolute differences and a gradient based measure together to define the total matching score in order to increase the robustness.
      • The weight w which balance the CSAD and CGRAD portions. It is determined by maxi-mizing the number of reliable corresponding pixel pairs that are filtered by performing cross-checking test.