SlideShare a Scribd company logo
1 of 17
Object Stereo
2014/3/25 Yichong Bai Object Stereo 1
Problem
• Image pair  Disparity map + Object level segmentation
• “A combined algorithm for stereo matching and object
segmentation”
2014/3/25 Yichong Bai Object Stereo 2
Overview
• Object level
• Segmentation in object level
• Model in object level
• Pixel level
• Pixel-corresponding needed
• 3D connectivity term
• Works with occlusion
• Slow, 20m/p
2014/3/25 Yichong Bai Object Stereo 3
Difference with ours
• Object level  too much semantics
• Initial disparity map needed
• Too slow
2014/3/25 Yichong Bai Object Stereo 4
Model
• Scene Representation
• Coarse to fine
• Object = object plane + parallax
• Parallax, surface-based representation
• Object
• Color model
• Parallax model
• Object plane
2014/3/25 Yichong Bai Object Stereo 5
Model
• Problem Formulation
• F: Pixel  Planes ( Depth)
• O: Pixel  Object
•
2014/3/25 Yichong Bai Object Stereo 6
Model
• Terms
• Photo Consistency
• Pixel dissimilarity
• Occlusion
• Corresponding pixels  same depth plane and object
•
•
2014/3/25 Yichong Bai Object Stereo 7
Model
• Terms
• Object Coherence
•
2014/3/25 Yichong Bai Object Stereo 8
Model
• Terms
• Depth Plane-Coherency
•
2014/3/25 Yichong Bai Object Stereo 9
Model
• Terms
• Object-Color
• GMM
• the probability that a pixel lies inside the object according to its color value
2014/3/25 Yichong Bai Object Stereo 10
Model
• Terms
• Object-Parallax
• regularizes the disparities of an object with respect to its object plane
• parallax is likely to be compact
• non-parametric histogram
•
• is histogram probability
2014/3/25 Yichong Bai Object Stereo 11
Model
• Terms
• Object-MDL
• Small no. of objects
2014/3/25 Yichong Bai Object Stereo 12
Model
• Terms
• 3D Connectivity Prior
• Connectivity = same object OR occluded
• Approximation:
• We randomly sample a large set of pairs of points p and q, where p and q belong
to different connected components of the object. We then draw a line between
p and q and check if all pixels on the line satisfy condition (10).
2014/3/25 Yichong Bai Object Stereo 13
Solve
• Energy Minimization
• Fusion move
• Proposal generator
• Like generics algorithm
• Proposal  Fuse  Evaluate
2014/3/25 Yichong Bai Object Stereo 14
Solve
• Proposals
• Initialization
• We first compute a disparity map F using the fast stereo matcher
• F is now derived by fitting a plane to each color segment using the initial
disparity map
• To derive O, we take the color segmentation result and group segments that
have similar depths according to F
• GMM: EM
• 30 Initial proposals, with difference segmentations and parameters
2014/3/25 Yichong Bai Object Stereo 15
Solve
• Iteration
• Refit
• Fix F, refine O
• Color model, Object plane, Parallax model
• Expansion
• derived by setting all pixels of F to f and all pixels of O to o
2014/3/25 Yichong Bai Object Stereo 16
Evaluation
• Middlebury benchmark
• Works best on cone set (artificial planar scene)
2014/3/25 Yichong Bai Object Stereo 17

More Related Content

Recently uploaded

Climate extremes likely to drive land mammal extinction during next supercont...
Climate extremes likely to drive land mammal extinction during next supercont...Climate extremes likely to drive land mammal extinction during next supercont...
Climate extremes likely to drive land mammal extinction during next supercont...
Sérgio Sacani
 
The solar dynamo begins near the surface
The solar dynamo begins near the surfaceThe solar dynamo begins near the surface
The solar dynamo begins near the surface
Sérgio Sacani
 
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Sérgio Sacani
 

Recently uploaded (20)

Climate extremes likely to drive land mammal extinction during next supercont...
Climate extremes likely to drive land mammal extinction during next supercont...Climate extremes likely to drive land mammal extinction during next supercont...
Climate extremes likely to drive land mammal extinction during next supercont...
 
The Scientific names of some important families of Industrial plants .pdf
The Scientific names of some important families of Industrial plants .pdfThe Scientific names of some important families of Industrial plants .pdf
The Scientific names of some important families of Industrial plants .pdf
 
family therapy psychotherapy types .pdf
family therapy psychotherapy types  .pdffamily therapy psychotherapy types  .pdf
family therapy psychotherapy types .pdf
 
SCHISTOSOMA HEAMATOBIUM life cycle .pdf
SCHISTOSOMA HEAMATOBIUM life cycle  .pdfSCHISTOSOMA HEAMATOBIUM life cycle  .pdf
SCHISTOSOMA HEAMATOBIUM life cycle .pdf
 
Emergent ribozyme behaviors in oxychlorine brines indicate a unique niche for...
Emergent ribozyme behaviors in oxychlorine brines indicate a unique niche for...Emergent ribozyme behaviors in oxychlorine brines indicate a unique niche for...
Emergent ribozyme behaviors in oxychlorine brines indicate a unique niche for...
 
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
Ostiguy & Panizza & Moffitt (eds.) - Populism in Global Perspective. A Perfor...
 
Land use land cover change analysis and detection of its drivers using geospa...
Land use land cover change analysis and detection of its drivers using geospa...Land use land cover change analysis and detection of its drivers using geospa...
Land use land cover change analysis and detection of its drivers using geospa...
 
ERTHROPOIESIS: Dr. E. Muralinath & R. Gnana Lahari
ERTHROPOIESIS: Dr. E. Muralinath & R. Gnana LahariERTHROPOIESIS: Dr. E. Muralinath & R. Gnana Lahari
ERTHROPOIESIS: Dr. E. Muralinath & R. Gnana Lahari
 
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
Extensive Pollution of Uranus and Neptune’s Atmospheres by Upsweep of Icy Mat...
 
MODERN PHYSICS_REPORTING_QUANTA_.....pdf
MODERN PHYSICS_REPORTING_QUANTA_.....pdfMODERN PHYSICS_REPORTING_QUANTA_.....pdf
MODERN PHYSICS_REPORTING_QUANTA_.....pdf
 
A Giant Impact Origin for the First Subduction on Earth
A Giant Impact Origin for the First Subduction on EarthA Giant Impact Origin for the First Subduction on Earth
A Giant Impact Origin for the First Subduction on Earth
 
The solar dynamo begins near the surface
The solar dynamo begins near the surfaceThe solar dynamo begins near the surface
The solar dynamo begins near the surface
 
GBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interactionGBSN - Microbiology (Unit 6) Human and Microbial interaction
GBSN - Microbiology (Unit 6) Human and Microbial interaction
 
TEST BANK for Organic Chemistry 6th Edition.pdf
TEST BANK for Organic Chemistry 6th Edition.pdfTEST BANK for Organic Chemistry 6th Edition.pdf
TEST BANK for Organic Chemistry 6th Edition.pdf
 
INSIGHT Partner Profile: Tampere University
INSIGHT Partner Profile: Tampere UniversityINSIGHT Partner Profile: Tampere University
INSIGHT Partner Profile: Tampere University
 
GBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
GBSN - Biochemistry (Unit 4) Chemistry of CarbohydratesGBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
GBSN - Biochemistry (Unit 4) Chemistry of Carbohydrates
 
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
Exomoons & Exorings with the Habitable Worlds Observatory I: On the Detection...
 
Plasma proteins_ Dr.Muralinath_Dr.c. kalyan
Plasma proteins_ Dr.Muralinath_Dr.c. kalyanPlasma proteins_ Dr.Muralinath_Dr.c. kalyan
Plasma proteins_ Dr.Muralinath_Dr.c. kalyan
 
RACEMIzATION AND ISOMERISATION completed.pptx
RACEMIzATION AND ISOMERISATION completed.pptxRACEMIzATION AND ISOMERISATION completed.pptx
RACEMIzATION AND ISOMERISATION completed.pptx
 
GBSN - Microbiology (Unit 7) Microbiology in Everyday Life
GBSN - Microbiology (Unit 7) Microbiology in Everyday LifeGBSN - Microbiology (Unit 7) Microbiology in Everyday Life
GBSN - Microbiology (Unit 7) Microbiology in Everyday Life
 

Featured

How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 

Featured (20)

Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
 

Object stereo, Slide for Bleyer 10 paper

  • 1. Object Stereo 2014/3/25 Yichong Bai Object Stereo 1
  • 2. Problem • Image pair  Disparity map + Object level segmentation • “A combined algorithm for stereo matching and object segmentation” 2014/3/25 Yichong Bai Object Stereo 2
  • 3. Overview • Object level • Segmentation in object level • Model in object level • Pixel level • Pixel-corresponding needed • 3D connectivity term • Works with occlusion • Slow, 20m/p 2014/3/25 Yichong Bai Object Stereo 3
  • 4. Difference with ours • Object level  too much semantics • Initial disparity map needed • Too slow 2014/3/25 Yichong Bai Object Stereo 4
  • 5. Model • Scene Representation • Coarse to fine • Object = object plane + parallax • Parallax, surface-based representation • Object • Color model • Parallax model • Object plane 2014/3/25 Yichong Bai Object Stereo 5
  • 6. Model • Problem Formulation • F: Pixel  Planes ( Depth) • O: Pixel  Object • 2014/3/25 Yichong Bai Object Stereo 6
  • 7. Model • Terms • Photo Consistency • Pixel dissimilarity • Occlusion • Corresponding pixels  same depth plane and object • • 2014/3/25 Yichong Bai Object Stereo 7
  • 8. Model • Terms • Object Coherence • 2014/3/25 Yichong Bai Object Stereo 8
  • 9. Model • Terms • Depth Plane-Coherency • 2014/3/25 Yichong Bai Object Stereo 9
  • 10. Model • Terms • Object-Color • GMM • the probability that a pixel lies inside the object according to its color value 2014/3/25 Yichong Bai Object Stereo 10
  • 11. Model • Terms • Object-Parallax • regularizes the disparities of an object with respect to its object plane • parallax is likely to be compact • non-parametric histogram • • is histogram probability 2014/3/25 Yichong Bai Object Stereo 11
  • 12. Model • Terms • Object-MDL • Small no. of objects 2014/3/25 Yichong Bai Object Stereo 12
  • 13. Model • Terms • 3D Connectivity Prior • Connectivity = same object OR occluded • Approximation: • We randomly sample a large set of pairs of points p and q, where p and q belong to different connected components of the object. We then draw a line between p and q and check if all pixels on the line satisfy condition (10). 2014/3/25 Yichong Bai Object Stereo 13
  • 14. Solve • Energy Minimization • Fusion move • Proposal generator • Like generics algorithm • Proposal  Fuse  Evaluate 2014/3/25 Yichong Bai Object Stereo 14
  • 15. Solve • Proposals • Initialization • We first compute a disparity map F using the fast stereo matcher • F is now derived by fitting a plane to each color segment using the initial disparity map • To derive O, we take the color segmentation result and group segments that have similar depths according to F • GMM: EM • 30 Initial proposals, with difference segmentations and parameters 2014/3/25 Yichong Bai Object Stereo 15
  • 16. Solve • Iteration • Refit • Fix F, refine O • Color model, Object plane, Parallax model • Expansion • derived by setting all pixels of F to f and all pixels of O to o 2014/3/25 Yichong Bai Object Stereo 16
  • 17. Evaluation • Middlebury benchmark • Works best on cone set (artificial planar scene) 2014/3/25 Yichong Bai Object Stereo 17