SlideShare a Scribd company logo
Recovering Transparent Shape
from Time-of-Flight Distortion
(CVPR2016)
1
K. Tanaka Y. Mukaigawa H. Kubo Y. Matsushita Y. Yagi
Transparent Objects
• Invisible, but distorted background can be seen.
• 3D reconstruction of transparent material is challenging.
3
Sensor
Estimated point
by triangulation
BackgroundReference
Distorted
Time-of-Flight (ToF) Camera
• Depth sensor based on time delay of light
• Kinect v2, Project Tango, etc.
4
time
Light signal
Observation
𝑡Δ
𝑑 =
𝑐𝑡Δ
2
(speed of light x time delay)
Time of Flight Distortion
• Speed of light slows down depending on refractive index.
• Depth becomes longer ( = ToF Distortion).
• We use this distortion for transparent shape recovery.
5
c
c
c
Contributions
6
1. ToF distortion can be used for transparent shape recovery.
2. Easy multi-path mitigation using retroreflective sheet.
c
c
c
Problem Setting
Input
• Known refractive index
• 1 distorted ToF depth
• 2 references (3D points)
7
r
f
b
r
t
v v
r
s
Output
• 3D points of both surfaces
• Surface normals
Parameters and Candidate Shapes
• Candidate shapes
• Front surface is on camera ray at distance 𝑡
• Back surface is on reference ray at distance 𝑠
• Many candidates. (2 degree of freedom)
8
ToF camera
𝑡
Glass object
Display or
known pattern
𝑠
Candidate Shape using ToF Distortion
• Candidate shapes
• Front surface is on camera ray at distance 𝑡
• Back surface is on reference ray at distance 𝑠
• such that 𝑠 + 𝑡 + 𝜂 = 𝑙 𝑇𝑜𝐹 : ToF distortion
• One degree of freedom
9
ToF camera
𝑡
Glass object
Display or
known pattern
𝑠
Surface Normal Consistency
• Surface normal is unique
Refractive normal
Geometric normal
• They should coincide.
10
ToF camera
𝑡
Glass object
Display or
known pattern
𝑠
𝑛 =
sin 𝜃1
sin 𝜃2
Refractive normal
camera ray
Geometric normal
Real world experiment setup
• modified Kinect v2 and LCD panel
11
Kinect v2
(IR Lens changed)
LCD panel
Linear stage
Target object
Results and evaluations
• Target materials and estimated results
• Evaluation
• Fit estimated points to ground-truth CAD model by ICP
12
Cube Wedge prism Schmidt prism
Object Mean error Std. dev.
Cube 0.188 mm 0.458 mm
Wedge 0.226 mm 1.137 mm
Schmidt 0.381 mm 1.398 mm
Summary
Input
• 1 distorted ToF depth
• 2 references (3D points)
• Known refractive index
13
r
f
b
r
t
v v
r
s
Output
• 3D points of both surfaces
• Surface normals
Time-of-Flight as alternative imager
• Light-in-flight [Gkioulekas+2015]
• Parameter tunable ToF camera (Texas instruments)
14
Imaging, Analyzing using ToF Camera
• Recently Emerging Topic
[Heide+2013], [Kadambi+2013], [Naik+2013], [Godbaz+2013], [Freedman+2014],
[Lin+2014], [O’Toole+2014], [Gupta+2015], [Heide+2015], [Xiao+2015],
[Kadambi+2015], [Peters+2015], [Tadano+2015], and more!
• CVPR 2016
• 1 oral, 2 posters (including ours)
[Kadambi et al.], [Su et al.]
• SIGGRAPH 2016
• 2 technical papers.
[Shrestha et al.], [Kadambi et al.]
15
We will continue working on ToF camera

More Related Content

What's hot

Jonathan Lefman presents his work on Superresolution chemical microscopy
Jonathan Lefman presents his work on Superresolution chemical microscopyJonathan Lefman presents his work on Superresolution chemical microscopy
Jonathan Lefman presents his work on Superresolution chemical microscopy
Jonathan Lefman
 
Evaluation of geometrical parameters of buildings from SAR images
Evaluation of geometrical parameters of buildings from SAR imagesEvaluation of geometrical parameters of buildings from SAR images
Evaluation of geometrical parameters of buildings from SAR images
Federico Ariu
 
Geometry Shader
Geometry ShaderGeometry Shader
Geometry Shader
acbess
 
Generation and weighting of 3D point correspondences for improved registratio...
Generation and weighting of 3D point correspondences for improved registratio...Generation and weighting of 3D point correspondences for improved registratio...
Generation and weighting of 3D point correspondences for improved registratio...
Kourosh Khoshelham
 
VJAI Paper Reading#3-KDD2019-ClusterGCN
VJAI Paper Reading#3-KDD2019-ClusterGCNVJAI Paper Reading#3-KDD2019-ClusterGCN
VJAI Paper Reading#3-KDD2019-ClusterGCN
Dat Nguyen
 
Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013
Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013
Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013
Sunando Sengupta
 
Tennis video shot classification based on support vector
Tennis video shot classification based on support vectorTennis video shot classification based on support vector
Tennis video shot classification based on support vector
es712
 
위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등
DACON AI 데이콘
 
Building Worlds - Codebits
Building Worlds - CodebitsBuilding Worlds - Codebits
Building Worlds - Codebits
Filipe Varela
 
A computationally efficient method for sequential MAP-MRF cloud detection
A computationally efficient method for sequential MAP-MRF cloud detectionA computationally efficient method for sequential MAP-MRF cloud detection
A computationally efficient method for sequential MAP-MRF cloud detection
Beniamino Murgante
 
An Analysis of Convolution for Inference
An Analysis of Convolution for InferenceAn Analysis of Convolution for Inference
An Analysis of Convolution for Inference
Intel Nervana
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
(Paper Review)3D shape reconstruction from sketches via multi view convolutio...
(Paper Review)3D shape reconstruction from sketches via multi view convolutio...(Paper Review)3D shape reconstruction from sketches via multi view convolutio...
(Paper Review)3D shape reconstruction from sketches via multi view convolutio...
MYEONGGYU LEE
 
Build Your Own 3D Scanner: Conclusion
Build Your Own 3D Scanner: ConclusionBuild Your Own 3D Scanner: Conclusion
Build Your Own 3D Scanner: Conclusion
Douglas Lanman
 
Real-Time Visual Simulation of Smoke
Real-Time Visual Simulation of SmokeReal-Time Visual Simulation of Smoke
Real-Time Visual Simulation of Smoke
Muhammad Karim
 
object detection paper review
object detection paper reviewobject detection paper review
object detection paper review
Yoonho Na
 
RasterFrames: Enabling Global-Scale Geospatial Machine Learning
RasterFrames: Enabling Global-Scale Geospatial Machine LearningRasterFrames: Enabling Global-Scale Geospatial Machine Learning
RasterFrames: Enabling Global-Scale Geospatial Machine Learning
Astraea, Inc.
 
Soft Shadow Maps for Linear Lights
Soft Shadow Maps for Linear LightsSoft Shadow Maps for Linear Lights
Soft Shadow Maps for Linear Lights
stefan_b
 
Point cloud mesh-investigation_report-lihang
Point cloud mesh-investigation_report-lihangPoint cloud mesh-investigation_report-lihang
Point cloud mesh-investigation_report-lihang
Lihang Li
 
BEFLIX
BEFLIXBEFLIX

What's hot (20)

Jonathan Lefman presents his work on Superresolution chemical microscopy
Jonathan Lefman presents his work on Superresolution chemical microscopyJonathan Lefman presents his work on Superresolution chemical microscopy
Jonathan Lefman presents his work on Superresolution chemical microscopy
 
Evaluation of geometrical parameters of buildings from SAR images
Evaluation of geometrical parameters of buildings from SAR imagesEvaluation of geometrical parameters of buildings from SAR images
Evaluation of geometrical parameters of buildings from SAR images
 
Geometry Shader
Geometry ShaderGeometry Shader
Geometry Shader
 
Generation and weighting of 3D point correspondences for improved registratio...
Generation and weighting of 3D point correspondences for improved registratio...Generation and weighting of 3D point correspondences for improved registratio...
Generation and weighting of 3D point correspondences for improved registratio...
 
VJAI Paper Reading#3-KDD2019-ClusterGCN
VJAI Paper Reading#3-KDD2019-ClusterGCNVJAI Paper Reading#3-KDD2019-ClusterGCN
VJAI Paper Reading#3-KDD2019-ClusterGCN
 
Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013
Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013
Urban 3D Semantic Modelling Using Stereo Vision, ICRA 2013
 
Tennis video shot classification based on support vector
Tennis video shot classification based on support vectorTennis video shot classification based on support vector
Tennis video shot classification based on support vector
 
위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등
 
Building Worlds - Codebits
Building Worlds - CodebitsBuilding Worlds - Codebits
Building Worlds - Codebits
 
A computationally efficient method for sequential MAP-MRF cloud detection
A computationally efficient method for sequential MAP-MRF cloud detectionA computationally efficient method for sequential MAP-MRF cloud detection
A computationally efficient method for sequential MAP-MRF cloud detection
 
An Analysis of Convolution for Inference
An Analysis of Convolution for InferenceAn Analysis of Convolution for Inference
An Analysis of Convolution for Inference
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
 
(Paper Review)3D shape reconstruction from sketches via multi view convolutio...
(Paper Review)3D shape reconstruction from sketches via multi view convolutio...(Paper Review)3D shape reconstruction from sketches via multi view convolutio...
(Paper Review)3D shape reconstruction from sketches via multi view convolutio...
 
Build Your Own 3D Scanner: Conclusion
Build Your Own 3D Scanner: ConclusionBuild Your Own 3D Scanner: Conclusion
Build Your Own 3D Scanner: Conclusion
 
Real-Time Visual Simulation of Smoke
Real-Time Visual Simulation of SmokeReal-Time Visual Simulation of Smoke
Real-Time Visual Simulation of Smoke
 
object detection paper review
object detection paper reviewobject detection paper review
object detection paper review
 
RasterFrames: Enabling Global-Scale Geospatial Machine Learning
RasterFrames: Enabling Global-Scale Geospatial Machine LearningRasterFrames: Enabling Global-Scale Geospatial Machine Learning
RasterFrames: Enabling Global-Scale Geospatial Machine Learning
 
Soft Shadow Maps for Linear Lights
Soft Shadow Maps for Linear LightsSoft Shadow Maps for Linear Lights
Soft Shadow Maps for Linear Lights
 
Point cloud mesh-investigation_report-lihang
Point cloud mesh-investigation_report-lihangPoint cloud mesh-investigation_report-lihang
Point cloud mesh-investigation_report-lihang
 
BEFLIX
BEFLIXBEFLIX
BEFLIX
 

Viewers also liked

Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...
Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...
Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...
Kenichiro Tanaka
 
mihara_iccp16_presentation
mihara_iccp16_presentationmihara_iccp16_presentation
mihara_iccp16_presentation
Hajime Mihara
 
次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -
次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -
次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -
Hironobu Fujiyoshi
 
Light Detection and Ranging Technology for Energy Sector
Light Detection and Ranging Technology for Energy Sector Light Detection and Ranging Technology for Energy Sector
Light Detection and Ranging Technology for Energy Sector
Canadian Association of Drilling Engineers
 
Light Detection And Ranging (useless in slideshare, must be downloaded to pow...
Light Detection And Ranging (useless in slideshare, must be downloaded to pow...Light Detection And Ranging (useless in slideshare, must be downloaded to pow...
Light Detection And Ranging (useless in slideshare, must be downloaded to pow...
Nina Tvenge
 
Pop Art
Pop ArtPop Art
Pop Art
educ5254
 
Track 4 session 8 - st dev con 2016 - time of flight
Track 4   session 8 - st dev con 2016 - time of flightTrack 4   session 8 - st dev con 2016 - time of flight
Track 4 session 8 - st dev con 2016 - time of flight
ST_World
 
Lidar
LidarLidar
(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning
Masahiro Suzuki
 
新しい分散実行の仕組み PROCESS WARPについて
新しい分散実行の仕組み PROCESS WARPについて新しい分散実行の仕組み PROCESS WARPについて
新しい分散実行の仕組み PROCESS WARPについて
祐司 伊藤
 
Diamond West 3D Laser Scanning Presentation
Diamond West 3D Laser Scanning PresentationDiamond West 3D Laser Scanning Presentation
Diamond West 3D Laser Scanning Presentation
dustinwoomer
 
Time of Flight Cameras - Refael Whyte
Time of Flight Cameras - Refael WhyteTime of Flight Cameras - Refael Whyte
Time of Flight Cameras - Refael Whyte
Camera Culture Group, MIT Media Lab
 
(DL hacks輪読)Bayesian Neural Network
(DL hacks輪読)Bayesian Neural Network(DL hacks輪読)Bayesian Neural Network
(DL hacks輪読)Bayesian Neural Network
Masahiro Suzuki
 
Meta-Learning with Memory Augmented Neural Network
Meta-Learning with Memory Augmented Neural NetworkMeta-Learning with Memory Augmented Neural Network
Meta-Learning with Memory Augmented Neural Network
Yusuke Watanabe
 
Track 4 session 1 - st dev con 2016 - body area network and sensor synchron...
Track 4   session 1 - st dev con 2016 - body area network and sensor synchron...Track 4   session 1 - st dev con 2016 - body area network and sensor synchron...
Track 4 session 1 - st dev con 2016 - body area network and sensor synchron...
ST_World
 
深層リカレントニューラルネットワークを用いた日本語述語項構造解析
深層リカレントニューラルネットワークを用いた日本語述語項構造解析深層リカレントニューラルネットワークを用いた日本語述語項構造解析
深層リカレントニューラルネットワークを用いた日本語述語項構造解析
Hiroki Ouchi
 
Sensors for Biometry and Recognition - 2016 Report by Yole Developpement
Sensors for Biometry and Recognition - 2016 Report by Yole DeveloppementSensors for Biometry and Recognition - 2016 Report by Yole Developpement
Sensors for Biometry and Recognition - 2016 Report by Yole Developpement
Yole Developpement
 
SlideShare moves to HTML5
SlideShare moves to HTML5 SlideShare moves to HTML5
SlideShare moves to HTML5
Amit Ranjan
 
NIPS読み会2013: One-shot learning by inverting a compositional causal process
NIPS読み会2013: One-shot learning by inverting  a compositional causal processNIPS読み会2013: One-shot learning by inverting  a compositional causal process
NIPS読み会2013: One-shot learning by inverting a compositional causal process
nozyh
 
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
StanfordComputationalImaging
 

Viewers also liked (20)

Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...
Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...
Recovering Inner Slices of Translucent Objects by Multi-frequency Illuminatio...
 
mihara_iccp16_presentation
mihara_iccp16_presentationmihara_iccp16_presentation
mihara_iccp16_presentation
 
次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -
次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -
次世代セキュリティを牽引する画像解析技術の最新動向 - 距離情報を用いた物体認識技術 -
 
Light Detection and Ranging Technology for Energy Sector
Light Detection and Ranging Technology for Energy Sector Light Detection and Ranging Technology for Energy Sector
Light Detection and Ranging Technology for Energy Sector
 
Light Detection And Ranging (useless in slideshare, must be downloaded to pow...
Light Detection And Ranging (useless in slideshare, must be downloaded to pow...Light Detection And Ranging (useless in slideshare, must be downloaded to pow...
Light Detection And Ranging (useless in slideshare, must be downloaded to pow...
 
Pop Art
Pop ArtPop Art
Pop Art
 
Track 4 session 8 - st dev con 2016 - time of flight
Track 4   session 8 - st dev con 2016 - time of flightTrack 4   session 8 - st dev con 2016 - time of flight
Track 4 session 8 - st dev con 2016 - time of flight
 
Lidar
LidarLidar
Lidar
 
(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning(DL輪読)Matching Networks for One Shot Learning
(DL輪読)Matching Networks for One Shot Learning
 
新しい分散実行の仕組み PROCESS WARPについて
新しい分散実行の仕組み PROCESS WARPについて新しい分散実行の仕組み PROCESS WARPについて
新しい分散実行の仕組み PROCESS WARPについて
 
Diamond West 3D Laser Scanning Presentation
Diamond West 3D Laser Scanning PresentationDiamond West 3D Laser Scanning Presentation
Diamond West 3D Laser Scanning Presentation
 
Time of Flight Cameras - Refael Whyte
Time of Flight Cameras - Refael WhyteTime of Flight Cameras - Refael Whyte
Time of Flight Cameras - Refael Whyte
 
(DL hacks輪読)Bayesian Neural Network
(DL hacks輪読)Bayesian Neural Network(DL hacks輪読)Bayesian Neural Network
(DL hacks輪読)Bayesian Neural Network
 
Meta-Learning with Memory Augmented Neural Network
Meta-Learning with Memory Augmented Neural NetworkMeta-Learning with Memory Augmented Neural Network
Meta-Learning with Memory Augmented Neural Network
 
Track 4 session 1 - st dev con 2016 - body area network and sensor synchron...
Track 4   session 1 - st dev con 2016 - body area network and sensor synchron...Track 4   session 1 - st dev con 2016 - body area network and sensor synchron...
Track 4 session 1 - st dev con 2016 - body area network and sensor synchron...
 
深層リカレントニューラルネットワークを用いた日本語述語項構造解析
深層リカレントニューラルネットワークを用いた日本語述語項構造解析深層リカレントニューラルネットワークを用いた日本語述語項構造解析
深層リカレントニューラルネットワークを用いた日本語述語項構造解析
 
Sensors for Biometry and Recognition - 2016 Report by Yole Developpement
Sensors for Biometry and Recognition - 2016 Report by Yole DeveloppementSensors for Biometry and Recognition - 2016 Report by Yole Developpement
Sensors for Biometry and Recognition - 2016 Report by Yole Developpement
 
SlideShare moves to HTML5
SlideShare moves to HTML5 SlideShare moves to HTML5
SlideShare moves to HTML5
 
NIPS読み会2013: One-shot learning by inverting a compositional causal process
NIPS読み会2013: One-shot learning by inverting  a compositional causal processNIPS読み会2013: One-shot learning by inverting  a compositional causal process
NIPS読み会2013: One-shot learning by inverting a compositional causal process
 
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
 

Similar to MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Tomoya Sato Master Thesis
Tomoya Sato Master ThesisTomoya Sato Master Thesis
Tomoya Sato Master Thesis
pflab
 
3DSensing.ppt
3DSensing.ppt3DSensing.ppt
3DSensing.ppt
TejaReddy453140
 
Computer Science Thesis Defense
Computer Science Thesis DefenseComputer Science Thesis Defense
Computer Science Thesis Defense
tompitkin
 
Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...
Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...
Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...
Jessica Ogden
 
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic..."What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
Edge AI and Vision Alliance
 
A Fast Implicit Gaussian Curvature Filter
A Fast Implicit Gaussian Curvature FilterA Fast Implicit Gaussian Curvature Filter
A Fast Implicit Gaussian Curvature Filter
Yuanhao Gong
 
高解析度面板瑕疵檢測
高解析度面板瑕疵檢測高解析度面板瑕疵檢測
高解析度面板瑕疵檢測
CHENHuiMei
 
Build Your Own 3D Scanner: 3D Scanning with Swept-Planes
Build Your Own 3D Scanner: 3D Scanning with Swept-PlanesBuild Your Own 3D Scanner: 3D Scanning with Swept-Planes
Build Your Own 3D Scanner: 3D Scanning with Swept-Planes
Douglas Lanman
 
DGPS
DGPSDGPS
Shiny Pixels and Beyond: Real-Time Raytracing at SEED
Shiny Pixels and Beyond: Real-Time Raytracing at SEEDShiny Pixels and Beyond: Real-Time Raytracing at SEED
Shiny Pixels and Beyond: Real-Time Raytracing at SEED
Electronic Arts / DICE
 
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to BlenderFrom STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
Emanuele Simioni
 
998-isvc16
998-isvc16998-isvc16
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Joo-Haeng Lee
 
[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...
[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...
[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...
thanhdowork
 
ZJPeng.3DSolderBallReconstruction
ZJPeng.3DSolderBallReconstructionZJPeng.3DSolderBallReconstruction
ZJPeng.3DSolderBallReconstruction
Zhejian Peng
 
Analysis of KinectFusion
Analysis of KinectFusionAnalysis of KinectFusion
Analysis of KinectFusion
Dong-Won Shin
 
Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...
Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...
Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...
NopphawanTamkuan
 
Indoor Navigation
Indoor NavigationIndoor Navigation
Indoor Navigation
Youssef Swellam
 
Crysis Next-Gen Effects (GDC 2008)
Crysis Next-Gen Effects (GDC 2008)Crysis Next-Gen Effects (GDC 2008)
Crysis Next-Gen Effects (GDC 2008)
Tiago Sousa
 
Dynamic shear stress evaluation on micro turning tool using photoelasticity
Dynamic shear stress evaluation on micro turning tool using photoelasticityDynamic shear stress evaluation on micro turning tool using photoelasticity
Dynamic shear stress evaluation on micro turning tool using photoelasticity
Soumen Mandal
 

Similar to MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016) (20)

Tomoya Sato Master Thesis
Tomoya Sato Master ThesisTomoya Sato Master Thesis
Tomoya Sato Master Thesis
 
3DSensing.ppt
3DSensing.ppt3DSensing.ppt
3DSensing.ppt
 
Computer Science Thesis Defense
Computer Science Thesis DefenseComputer Science Thesis Defense
Computer Science Thesis Defense
 
Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...
Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...
Highs and Lows: A Resel-based Approach to the Analysis of Data from Geophysic...
 
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic..."What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
 
A Fast Implicit Gaussian Curvature Filter
A Fast Implicit Gaussian Curvature FilterA Fast Implicit Gaussian Curvature Filter
A Fast Implicit Gaussian Curvature Filter
 
高解析度面板瑕疵檢測
高解析度面板瑕疵檢測高解析度面板瑕疵檢測
高解析度面板瑕疵檢測
 
Build Your Own 3D Scanner: 3D Scanning with Swept-Planes
Build Your Own 3D Scanner: 3D Scanning with Swept-PlanesBuild Your Own 3D Scanner: 3D Scanning with Swept-Planes
Build Your Own 3D Scanner: 3D Scanning with Swept-Planes
 
DGPS
DGPSDGPS
DGPS
 
Shiny Pixels and Beyond: Real-Time Raytracing at SEED
Shiny Pixels and Beyond: Real-Time Raytracing at SEEDShiny Pixels and Beyond: Real-Time Raytracing at SEED
Shiny Pixels and Beyond: Real-Time Raytracing at SEED
 
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to BlenderFrom STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
 
998-isvc16
998-isvc16998-isvc16
998-isvc16
 
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
 
[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...
[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...
[20240603_LabSeminar_Huy]TransMOT: Spatial-Temporal Graph Transformer for Mul...
 
ZJPeng.3DSolderBallReconstruction
ZJPeng.3DSolderBallReconstructionZJPeng.3DSolderBallReconstruction
ZJPeng.3DSolderBallReconstruction
 
Analysis of KinectFusion
Analysis of KinectFusionAnalysis of KinectFusion
Analysis of KinectFusion
 
Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...
Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...
Data Processing Using DubaiSat Satellite Imagery for Disaster Monitoring (Cas...
 
Indoor Navigation
Indoor NavigationIndoor Navigation
Indoor Navigation
 
Crysis Next-Gen Effects (GDC 2008)
Crysis Next-Gen Effects (GDC 2008)Crysis Next-Gen Effects (GDC 2008)
Crysis Next-Gen Effects (GDC 2008)
 
Dynamic shear stress evaluation on micro turning tool using photoelasticity
Dynamic shear stress evaluation on micro turning tool using photoelasticityDynamic shear stress evaluation on micro turning tool using photoelasticity
Dynamic shear stress evaluation on micro turning tool using photoelasticity
 

Recently uploaded

Physiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptxPhysiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptx
fatima132662
 
8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf
by6843629
 
11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf
PirithiRaju
 
AJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdfAJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
Leonel Morgado
 
Summary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdfSummary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdf
vadgavevedant86
 
fermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptxfermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptx
ananya23nair
 
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Creative-Biolabs
 
Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...
Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...
Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...
frank0071
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
sammy700571
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
hozt8xgk
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
Frédéric Baudron
 
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
Sérgio Sacani
 
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
Sérgio Sacani
 
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
Sérgio Sacani
 
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Leonel Morgado
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
RDhivya6
 
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
ABHISHEK SONI NIMT INSTITUTE OF MEDICAL AND PARAMEDCIAL SCIENCES , GOVT PG COLLEGE NOIDA
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
Vandana Devesh Sharma
 
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
Advanced-Concepts-Team
 

Recently uploaded (20)

Physiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptxPhysiology of Nervous System presentation.pptx
Physiology of Nervous System presentation.pptx
 
8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf
 
11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf11.1 Role of physical biological in deterioration of grains.pdf
11.1 Role of physical biological in deterioration of grains.pdf
 
AJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdfAJAY KUMAR NIET GreNo Guava Project File.pdf
AJAY KUMAR NIET GreNo Guava Project File.pdf
 
Immersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths ForwardImmersive Learning That Works: Research Grounding and Paths Forward
Immersive Learning That Works: Research Grounding and Paths Forward
 
Summary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdfSummary Of transcription and Translation.pdf
Summary Of transcription and Translation.pdf
 
fermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptxfermented food science of sauerkraut.pptx
fermented food science of sauerkraut.pptx
 
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
Mechanisms and Applications of Antiviral Neutralizing Antibodies - Creative B...
 
Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...
Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...
Juaristi, Jon. - El canon espanol. El legado de la cultura española a la civi...
 
Microbiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdfMicrobiology of Central Nervous System INFECTIONS.pdf
Microbiology of Central Nervous System INFECTIONS.pdf
 
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
快速办理(UAM毕业证书)马德里自治大学毕业证学位证一模一样
 
Farming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptxFarming systems analysis: what have we learnt?.pptx
Farming systems analysis: what have we learnt?.pptx
 
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...Discovery of An Apparent Red, High-Velocity Type Ia Supernova at  𝐳 = 2.9  wi...
Discovery of An Apparent Red, High-Velocity Type Ia Supernova at 𝐳 = 2.9 wi...
 
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
SDSS1335+0728: The awakening of a ∼ 106M⊙ black hole⋆
 
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDSJAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
JAMES WEBB STUDY THE MASSIVE BLACK HOLE SEEDS
 
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...
 
23PH301 - Optics - Optical Lenses.pptx
23PH301 - Optics  -  Optical Lenses.pptx23PH301 - Optics  -  Optical Lenses.pptx
23PH301 - Optics - Optical Lenses.pptx
 
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
MICROBIAL INTERACTION PPT/ MICROBIAL INTERACTION AND THEIR TYPES // PLANT MIC...
 
Compexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titrationCompexometric titration/Chelatorphy titration/chelating titration
Compexometric titration/Chelatorphy titration/chelating titration
 
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
ESA/ACT Science Coffee: Diego Blas - Gravitational wave detection with orbita...
 

MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

  • 1. Recovering Transparent Shape from Time-of-Flight Distortion (CVPR2016) 1 K. Tanaka Y. Mukaigawa H. Kubo Y. Matsushita Y. Yagi
  • 2. Transparent Objects • Invisible, but distorted background can be seen. • 3D reconstruction of transparent material is challenging. 3 Sensor Estimated point by triangulation BackgroundReference Distorted
  • 3. Time-of-Flight (ToF) Camera • Depth sensor based on time delay of light • Kinect v2, Project Tango, etc. 4 time Light signal Observation 𝑡Δ 𝑑 = 𝑐𝑡Δ 2 (speed of light x time delay)
  • 4. Time of Flight Distortion • Speed of light slows down depending on refractive index. • Depth becomes longer ( = ToF Distortion). • We use this distortion for transparent shape recovery. 5 c c c
  • 5. Contributions 6 1. ToF distortion can be used for transparent shape recovery. 2. Easy multi-path mitigation using retroreflective sheet. c c c
  • 6. Problem Setting Input • Known refractive index • 1 distorted ToF depth • 2 references (3D points) 7 r f b r t v v r s Output • 3D points of both surfaces • Surface normals
  • 7. Parameters and Candidate Shapes • Candidate shapes • Front surface is on camera ray at distance 𝑡 • Back surface is on reference ray at distance 𝑠 • Many candidates. (2 degree of freedom) 8 ToF camera 𝑡 Glass object Display or known pattern 𝑠
  • 8. Candidate Shape using ToF Distortion • Candidate shapes • Front surface is on camera ray at distance 𝑡 • Back surface is on reference ray at distance 𝑠 • such that 𝑠 + 𝑡 + 𝜂 = 𝑙 𝑇𝑜𝐹 : ToF distortion • One degree of freedom 9 ToF camera 𝑡 Glass object Display or known pattern 𝑠
  • 9. Surface Normal Consistency • Surface normal is unique Refractive normal Geometric normal • They should coincide. 10 ToF camera 𝑡 Glass object Display or known pattern 𝑠 𝑛 = sin 𝜃1 sin 𝜃2 Refractive normal camera ray Geometric normal
  • 10. Real world experiment setup • modified Kinect v2 and LCD panel 11 Kinect v2 (IR Lens changed) LCD panel Linear stage Target object
  • 11. Results and evaluations • Target materials and estimated results • Evaluation • Fit estimated points to ground-truth CAD model by ICP 12 Cube Wedge prism Schmidt prism Object Mean error Std. dev. Cube 0.188 mm 0.458 mm Wedge 0.226 mm 1.137 mm Schmidt 0.381 mm 1.398 mm
  • 12. Summary Input • 1 distorted ToF depth • 2 references (3D points) • Known refractive index 13 r f b r t v v r s Output • 3D points of both surfaces • Surface normals
  • 13. Time-of-Flight as alternative imager • Light-in-flight [Gkioulekas+2015] • Parameter tunable ToF camera (Texas instruments) 14
  • 14. Imaging, Analyzing using ToF Camera • Recently Emerging Topic [Heide+2013], [Kadambi+2013], [Naik+2013], [Godbaz+2013], [Freedman+2014], [Lin+2014], [O’Toole+2014], [Gupta+2015], [Heide+2015], [Xiao+2015], [Kadambi+2015], [Peters+2015], [Tadano+2015], and more! • CVPR 2016 • 1 oral, 2 posters (including ours) [Kadambi et al.], [Su et al.] • SIGGRAPH 2016 • 2 technical papers. [Shrestha et al.], [Kadambi et al.] 15 We will continue working on ToF camera