Your SlideShare is downloading. ×
0
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Disparity Estimation Using A Color Segmentation V3
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Disparity Estimation Using A Color Segmentation V3

1,260

Published on

Master_Project

Master_Project

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,260
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Disparity Estimation using a Color Segmentation Xin Wang Barcelona, 3rd Sep. 2009
  • 2. Outline for the presentation <ul><li>Task and Problem Identification </li></ul><ul><li>Algorithmic Overview </li></ul><ul><li>Segmentation-based Stereo Matching </li></ul><ul><ul><li>Local Matching </li></ul></ul><ul><ul><li>Segmentation-based Disparity Fitting </li></ul></ul><ul><ul><li>Disparity Refinement via Region Merging </li></ul></ul><ul><li>Experiment Result and Analysis </li></ul><ul><li>Conclusions and Future Work </li></ul><ul><li>Questions </li></ul>
  • 3. Task and Problem Identification <ul><li>Stereo Correspondence Problem </li></ul><ul><li>Given an element in the left image, we search for the corresponding element </li></ul><ul><li>in the right image. </li></ul><ul><ul><li>Disparity is the displacement for pixel positions between two corresponding points in these images. </li></ul></ul><ul><ul><li>Epipolar Rectification allows simplifying the search problem from </li></ul></ul><ul><ul><li>2D to 1D. </li></ul></ul>
  • 4. Task and Problem Identification <ul><li>Challenges </li></ul><ul><ul><li>Textureless Region : Matching in an untextured area becomes ambiguous, since there are many potential matching points of very similar intensity patterns. </li></ul></ul><ul><ul><li>Depth Discontinuity Region : Pixels whose neighbouring disparities differ by more than a given value, usually appear at the boundaries of different scene objects. </li></ul></ul><ul><ul><li>Occluded Region : Zones that are occluded in the matching image. </li></ul></ul>
  • 5. Task and Problem Identification <ul><li>Quality Measure : </li></ul><ul><ul><li>Reconstruction : </li></ul></ul><ul><ul><li>To reconstruct the reference image in a back-ward way, is using </li></ul></ul><ul><ul><li>reference image’s disparity map and the other view image for </li></ul></ul><ul><ul><li>reconstruction. </li></ul></ul><ul><ul><li>Reconstruction Image PSNR (Peak Signal to Noise Ration) : </li></ul></ul><ul><ul><li>, where </li></ul></ul>L (ori) R (ori) Disparity (Left) Disparity (Right)
  • 6. Task and Problem Identification <ul><ul><li>Occlusion mask is made from performing L-R check to groundtruth. </li></ul></ul><ul><ul><li>PSNR is Calculated in non-occluded region. </li></ul></ul>
  • 7. Outline for the presentation <ul><li>Task and Problem Identification </li></ul><ul><li>Algorithmic Overview </li></ul><ul><li>Segmentation-based Stereo Matching </li></ul><ul><ul><li>Local Matching </li></ul></ul><ul><ul><li>Segmentation-based Disparity Fitting </li></ul></ul><ul><ul><li>Disparity Refinement via Region Merging </li></ul></ul><ul><li>Experiment Result and Analysis </li></ul><ul><li>Conclusions and Future Work </li></ul><ul><li>Questions </li></ul>
  • 8. Algorithmic Overview <ul><li>Block Diagram of Algorithm </li></ul>Initial Disparity Estimation Color Segmentation to extract the label Polynomial Model based Disparity Representation in Each Color Segment (use label information) Disparity Improvement By Region Merging Iterations Final Disparity Map
  • 9. Outline for the presentation <ul><li>Task and Problem Identification </li></ul><ul><li>Algorithmic Overview </li></ul><ul><li>Segmentation-based Stereo Matching </li></ul><ul><ul><li>Local Matching </li></ul></ul><ul><ul><li>Segmentation-based Disparity Fitting </li></ul></ul><ul><ul><li>Disparity Refinement via Region Merging </li></ul></ul><ul><li>Experiment Result and Analysis </li></ul><ul><li>Conclusions and Future Work </li></ul><ul><li>Questions </li></ul>
  • 10. Initial Disparity Estimation <ul><li>Window-based Local Matching : </li></ul><ul><ul><li>Matching Score : is a quantitative similarity measurement for two different </li></ul></ul><ul><li>pixels. It is the criterion for the solution of correspondence problem. </li></ul><ul><ul><li>Winner-take-all : disparity associated with the minimum cost value is </li></ul></ul><ul><ul><li>selected at each pixel. </li></ul></ul>
  • 11. Outline for the presentation <ul><li>Task and Problem Identification </li></ul><ul><li>Algorithmic Overview </li></ul><ul><li>Segmentation-based Stereo Matching </li></ul><ul><ul><li>Local Matching </li></ul></ul><ul><ul><li>Segmentation-based Disparity Fitting </li></ul></ul><ul><ul><li>Disparity Refinement via Region Merging </li></ul></ul><ul><li>Experiment Result and Analysis </li></ul><ul><li>Conclusions and Future Work </li></ul><ul><li>Questions </li></ul>
  • 12. Segmentation-based Approach <ul><li>Approach: Segmentation-based approaches divide one or sometimes both images into non-overlapping regions of homogeneous color. </li></ul><ul><li>Assumptions: </li></ul><ul><ul><li>1, Inside a segment of homogeneous colour the disparity values are expected to follow some particular smooth disparity model (constant disparity, planar model, etc) </li></ul></ul><ul><ul><li>2, Disparity discontinuities are assumed to coincide with the boundaries </li></ul></ul><ul><ul><li>of those color regions. </li></ul></ul>
  • 13. Segmentation-based Approach <ul><li>Polynomial Models for Each Segment </li></ul><ul><ul><li>Order 0 Polynomial Model : </li></ul></ul><ul><ul><li>Order 1 Polynomial Model : </li></ul></ul><ul><ul><li>Order 2 Polynomial Model : </li></ul></ul>(planar parallel to the camera front plane) (planar model) (parabolic curved surface)
  • 14. Segmentation-based Approach <ul><li>Model Parameters Estimation </li></ul><ul><ul><li>We use initial disparity as input data </li></ul></ul><ul><ul><li>Unknown Parameters: </li></ul></ul><ul><ul><ul><li>Order 0 : 1 parameter - > 1 point </li></ul></ul></ul><ul><ul><ul><li>Order 1 : 3 parameters -> 3 points </li></ul></ul></ul><ul><ul><ul><li>Order 2 : 6 parameters -> 6 points </li></ul></ul></ul><ul><ul><li>Proposed Approaches for Model Fitting: </li></ul></ul><ul><ul><ul><li>Least Squares </li></ul></ul></ul><ul><ul><ul><li>Random Sample and Consensus (RANSAC) </li></ul></ul></ul>
  • 15. Segmentation-based Approach <ul><li>Least Squares </li></ul><ul><ul><li>Over-determined Problem : The number of points in each region is much higher than the number of equations needed to compute the analytical solution. </li></ul></ul><ul><ul><li>Best Fit for Observed Data : Sum of squared residuals has its least value. (but sensitive to outliers) </li></ul></ul><ul><ul><li>min( ) </li></ul></ul>
  • 16. Segmentation-based Approach <ul><li>RANSAC </li></ul><ul><li>Non-deterministic algorithm in the sense that it produces a </li></ul><ul><li>reasonable result only with a certain probability. </li></ul><ul><ul><li>Assumption : A set of data is that the data consists of inliers and the data's contribution follows a certain set of model parameters, while the outliers do not fit the model. </li></ul></ul><ul><ul><li>Algorithm: Hypothesize-and-Test framework </li></ul></ul><ul><ul><ul><li>HYPOTHESIZE: Minimal Sample Set (MSS) is randomly selected from initial input data. </li></ul></ul></ul><ul><ul><ul><li>TEST: Check the entire dataset with the estimated model from MSS. Data are consistent with the model are called consensus set (CS). </li></ul></ul></ul>
  • 17. Segmentation-based Approach <ul><li>RANSAC in Polynomial Model Fitting </li></ul><ul><ul><li>Polynomial Order 0 : Median filter </li></ul></ul><ul><ul><li>Polynomial Order 1 and 2: </li></ul></ul><ul><ul><ul><li>1. Select MSS : Randomly select 3 or 6 points from initial disparity points </li></ul></ul></ul><ul><ul><ul><li>2. Fit the Model by Solving Linear Equations System: </li></ul></ul></ul><ul><ul><ul><li>3. Select Inliers to Form the CS: </li></ul></ul></ul><ul><li>Distance calculation -> D = d i – f (x i ,y i ,P) if below the distance </li></ul><ul><li>threshold </li></ul><ul><ul><ul><li>4. Choose the best CS to Output Fitted Model: CS with most inliers </li></ul></ul></ul>
  • 18. Segmentation-based Approach <ul><li>Comparison between RANSAC & LS </li></ul><ul><ul><li>LS : Analytical solution, non-iterative solution. Less computation cost. </li></ul></ul><ul><ul><li>Very sensitive to outliers. </li></ul></ul><ul><ul><li>RANSAC: Robust to outliers, non-deterministic algorithm. Accuracy </li></ul></ul><ul><ul><li>will increase with increasing the iterations and strictness </li></ul></ul><ul><ul><li>in thresholds. </li></ul></ul><ul><li>More computation cost and the model may subject to </li></ul><ul><li>noise. </li></ul><ul><ul><li>PSNR Comparison Test: </li></ul></ul>(Teddy 559 regions, Cones 553 regions, Venus 398 regions)
  • 19. Outline for the presentation <ul><li>Task and Problem Identification </li></ul><ul><li>Algorithmic Overview </li></ul><ul><li>Segmentation-based Stereo Matching </li></ul><ul><ul><li>Local Matching </li></ul></ul><ul><ul><li>Segmentation-based Disparity Fitting </li></ul></ul><ul><ul><li>Disparity Refinement via Region Merging </li></ul></ul><ul><li>Experiment Result and Analysis </li></ul><ul><li>Conclusions and Future Work </li></ul><ul><li>Questions </li></ul>
  • 20. Disparity Refinement via Merging <ul><li>Problems : In some regions, due to unproper-segmentation, </li></ul><ul><li>occlusion, noise, lack of enough inliers, may result in </li></ul><ul><li>incorrect model estimation. </li></ul><ul><li>Solution : Merge adjacent “good regions” to re-fit the model. </li></ul><ul><ul><li>For lack of inliers region : enlarge the area , includes more inliers </li></ul></ul><ul><ul><li>For occlusion area : get the support from neighbouring good region </li></ul></ul><ul><ul><li>For small region : large distance separated points, more easy to fit a </li></ul></ul><ul><ul><li>correct model </li></ul></ul>
  • 21. Disparity Refinement via Merging <ul><li>Merging Block Diagram : </li></ul>
  • 22. Disparity Refinement via Merging <ul><li>Region Similarity Measure Criteria </li></ul><ul><ul><li>Similarity in Model’s Parameters </li></ul></ul><ul><ul><li>Number of inliers </li></ul></ul><ul><ul><li>PSNR of Reconstructed Region </li></ul></ul>(Euclidean Distance Based)
  • 23. Disparity Refinement via Merging <ul><li>Implementation for similarity measure: </li></ul><ul><ul><li>Distance similarity in region’s parameters + PSNR criterion </li></ul></ul><ul><ul><li>Distance similarity matrix will be calculated: </li></ul></ul><ul><ul><li>Due to distance similarity based on region’s parameters can not always be true, so I use PSNR criterion for actual merging step to secure merging process will always improve the disparity quality </li></ul></ul>
  • 24. Outline for the presentation <ul><li>Task and Problem Identification </li></ul><ul><li>Algorithmic Overview </li></ul><ul><li>Segmentation-based Stereo Matching </li></ul><ul><ul><li>Local Matching </li></ul></ul><ul><ul><li>Segmentation-based Disparity Fitting </li></ul></ul><ul><ul><li>Disparity Refinement via Region Merging </li></ul></ul><ul><li>Experiment Result and Analysis </li></ul><ul><li>Conclusions and Future Work </li></ul><ul><li>Questions </li></ul>
  • 25. Experiment Results <ul><li>Polynomial Model Fitting </li></ul><ul><ul><li>Compare with initial disparity, usually no improvement </li></ul></ul><ul><ul><li>Analysis : RANSAC works not well, due to initial color segmentation fails to meet the two assumptions, and some regions lack of enough inliers. </li></ul></ul>Parameters : Teddy (559 regions) , Cones (553 regions), Venus (398 regions) RANSAC iterations:200, inlier threshold:0.99 , distance threshold:0.5
  • 26. Experiment Results <ul><li>Mixed Order Polynomial Model Fitting </li></ul>Program runs on a 3GHz processor
  • 27. Experiment Results <ul><li>Mixed Order Polynomial Model Fitting </li></ul>Red: Order 0, Green: Order 1, Blue: Order 2, Black: No improvement
  • 28. Experiment Results <ul><li>Polynomial order 2 Model Region Merging </li></ul>Teddy (559 regions) , Cones (553 regions), Venus (398 regions)
  • 29. Experiment Results <ul><li>Polynomial order 2 Teddy case </li></ul>
  • 30. Experiment Results <ul><li>Polynomial order 1 model results: </li></ul>Reconstruction using initial Disparity Reconstruction using merged disparity, 160 rounds
  • 31. Conclusions and Future Work <ul><li>Conclusions: </li></ul><ul><ul><li>LS is more sensitive to outliers, RANSAC is more robust </li></ul></ul><ul><ul><li>Sometimes RANSAC will fail to estimate correct parameters, due to the segmented region’s size, noise etc. </li></ul></ul><ul><ul><li>Small regions may not have enough inliers, region merging could help </li></ul></ul><ul><ul><li>Bad estimated regions could get support from neighbouring regions in merging process </li></ul></ul><ul><li>Future work </li></ul><ul><ul><li>More accurate and computationally efficient region similarity measure should be further studied </li></ul></ul><ul><ul><li>Try to do merging for all the possibilities for adjacent regions. </li></ul></ul><ul><ul><li>Convert Matlab code to C++ version </li></ul></ul>
  • 32. Acknowledgement <ul><li>Prof. Josep Ramon Morros i Rubio </li></ul><ul><li>Albert (Technical support) </li></ul><ul><li>All Professors and friends in TSC who </li></ul><ul><li>supported me before. </li></ul>
  • 33. Questions <ul><li>Thank you very much ! </li></ul><ul><li>Questions? </li></ul>
  • 34. Experiment Results <ul><li>Example of Region Merging Improvement (Teddy, order2) </li></ul>A, improvement point B, how label changes C, disparity refinement
  • 35. Experiment Results <ul><li>Drop point in the evolution curve (214 & 216region) </li></ul>
  • 36. Experiment Results <ul><li>Order 1 Merging -> </li></ul><ul><li>Teddy </li></ul>
  • 37. Experiment Results <ul><li>Poor Modeled Region in order2 Teddy </li></ul>
  • 38. Disparity & Depth <ul><li>Disparity is proportional inversely to the depth </li></ul>
  • 39. Experiment Results <ul><li>Local Matching </li></ul><ul><ul><li>How the local window size affect local matching </li></ul></ul><ul><ul><li>Too small window can not catch enough intensity variation to give the correct disparity in less-textured regions, too large window can not capture well small details </li></ul></ul>Teddy Image Cones Image Venus Image
  • 40. Matching Score <ul><li>Matching Score: Matching measures such as SSD and SAD are strictly assuming the constant color con-straint. So if the computed area fails to meet this constraint, SSD and SAD measures may not be robust. While the other matching scores like gradient-based measure is more robust to changes in camera gain. So we could expect to combine the sum of absolute differences and a gradient based measure together to define the total matching score in order to increase the robustness. </li></ul><ul><li>The weight w which balance the CSAD and CGRAD portions. It is determined by maxi-mizing the number of reliable corresponding pixel pairs that are filtered by performing cross-checking test. </li></ul>

×