Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to PR-228: Geonet: Unsupervised learning of dense depth, optical flow and camera pose
Similar to PR-228: Geonet: Unsupervised learning of dense depth, optical flow and camera pose (20)
More from Hyeongmin Lee
More from Hyeongmin Lee (20)
Recently uploaded
Recently uploaded (20)
PR-228: Geonet: Unsupervised learning of dense depth, optical flow and camera pose
- 1. GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose Hyeongmin Lee Image and Video Pattern Recognition LAB Electrical and Electronic Engineering Dept, Yonsei University 5th Semester 2020.2.23
- 2. Depth, Optical Flow, Camera Pose
- 3. Depth, Optical Flow, Camera Pose ◆ Depth [PR098 - MegaDepth] 이미지에 등장하는 각 Pixel이 Camera로부터 몇 m 떨어져 있는지를 나타내는 Map
- 4. Depth, Optical Flow, Camera Pose ◆ Optical Flow [PR214 - FlowNet] 연속한 두 Frame 사이에서 각 Pixel의 Motion을 나타내는 Vector Map (Pixel Displacement)
- 5. Depth, Optical Flow, Camera Pose ◆ Camera Pose (Camera Motion, Ego-Motion) 𝑧 𝑥 𝑦 (𝑥, 𝑦, 𝑧) (0,0,0) (𝑥, 𝑦, 𝑧) (𝑥′, 𝑦′, 𝑧′) 𝑇
- 6. Depth, Optical Flow, Camera Pose ◆ Depth, Optical Flow, Camera Pose 대부분의 Pixel Motion은 카메라의 움직임에 의해 발생 ➔ Object Motion과 분리하여 생각.
- 7. Depth, Optical Flow, Camera Pose ◆ Depth, Optical Flow, Camera Pose Depth!!
- 8. 3D Geometry
- 9. 3D Geometry ◆ Real Distance? Camera 정보 카메라와 대상 간의 거리 (Depth)
- 10. 3D Geometry ◆ Camera Calibration Image Coordinate Normalized Coordinate pixel Meter(z=1) (𝑥, 𝑦) (𝑢, 𝑣) 𝑥 = 𝑓𝑥 𝑢 + 𝑐 𝑥 𝑦 = 𝑓𝑦 𝑣 + 𝑐 𝑦 𝑥 𝑦 1 = 𝑓𝑥 0 𝑐 𝑥 0 𝑓𝑦 𝑐 𝑦 0 0 1 𝑢 𝑣 1 𝐾 Intrinsic Parameter
- 11. 3D Geometry ◆ Depth 초점 𝑍 (𝑋, 𝑌, 𝑍) 1 𝑓 (𝑢, 𝑣, 1) (𝑥, 𝑦, 1) 𝑢 𝑣 1 = 𝐾−1 𝑥 𝑦 1 𝑋 𝑌 𝑍 = 𝑍 𝑢 𝑣 1 = 𝐷𝐾−1 𝑥 𝑦 1
- 12. 3D Geometry ◆ 3D Transformation (𝑥, 𝑦, 𝑧) (𝑥′, 𝑦′, 𝑧′) 𝑇 𝑥′ 𝑦′ 𝑧′ 1 = 𝑟11 𝑟12 𝑟13 𝑡 𝑥 𝑟11 𝑟12 𝑟13 𝑡 𝑥 𝑟11 𝑟12 𝑟13 𝑡 𝑥 0 0 0 1 𝑥 𝑦 𝑧 1 = [𝑅|𝑡] 𝑥 𝑦 𝑧 1 𝑥′ 𝑦′ 𝑧′ = 𝑅 𝑥 𝑦 𝑧 + 𝑡 𝑥 𝑡 𝑦 𝑡 𝑧 출처: Dark Programmer
- 13. GeoNet
- 14. GeoNet ◆ Rigid & Residual Motion • Rigid Motion: Camera Motion에 의한 상대적인 움직임 • Residual Motion: 각 Object의 독립적인 움직임
- 15. GeoNet ◆ Rigid & Residual Motion =
- 16. GeoNet ◆ Rigid Warping Loss ◆ Edge-Aware Depth Smoothness Loss 𝐿 𝑟𝑤 = 𝛼 1 − 𝑆𝑆𝐼𝑀(𝐼𝑡, ෩𝐼𝑠 𝑟𝑖𝑔 ) 2 + 1 − 𝛼 𝐼𝑡 − ෩𝐼𝑠 𝑟𝑖𝑔 1 𝐿 𝑑𝑠 = 𝑝 𝑡 |∇𝐷(𝑝𝑡)| ∙ 𝑒− ∇𝐼 𝑝 𝑡 𝑇
- 17. GeoNet ◆ Flow Warping Loss ◆ Edge-Aware Flow Smoothness Loss 𝐿 𝑓𝑤 = 𝛼 1 − 𝑆𝑆𝐼𝑀(𝐼𝑡, ෩𝐼𝑠 𝑓𝑢𝑙𝑙 ) 2 + 1 − 𝛼 𝐼𝑡 − ෩𝐼𝑠 𝑓𝑢𝑙𝑙 1 𝐿 𝑓𝑠 = 𝑝 𝑡 |∇𝑓𝑡→𝑠 𝑓𝑢𝑙𝑙 (𝑝𝑡)| ∙ 𝑒− ∇𝐼 𝑝 𝑡 𝑇
- 18. GeoNet ◆ Geometric Consistency Loss 𝐿 𝑔𝑐 = 𝑝 𝑡 [𝛿(𝑝𝑡)] ∙ ∆𝑓𝑡→𝑠 𝑓𝑢𝑙𝑙 𝑝𝑡 1 ∆𝑓𝑡→𝑠 𝑓𝑢𝑙𝑙 𝑝𝑡 = 𝑓𝑡→𝑠 𝑓𝑢𝑙𝑙 + 𝑓𝑠→𝑡 𝑓𝑢𝑙𝑙 (𝑝𝑡 + 𝑓𝑡→𝑠 𝑓𝑢𝑙𝑙 (𝑝𝑡)) For Occlusion Reasoning
- 20. GeoNet ◆ Flow & Pose Result
- 21. Thank You!