Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to KaziInjamamul_MasterThesis(UNITN & TU Delft)
Similar to KaziInjamamul_MasterThesis(UNITN & TU Delft) (20)
Recently uploaded
Recently uploaded (20)
KaziInjamamul_MasterThesis(UNITN & TU Delft)
- 1. Dynamic object reconstruction from multiple non-static view videos for free viewpoint animation Kazi Injamamul Haque Student ID: 204873 Supervisors: Prof. Elisa Ricci Dr. Fabio Poiesi Department of Information Engineering and Computer Science (DISI) Information and Communication Engineering
- 2. Video 2
- 4. 4 Solution A Dynamic Object Reconstruction Framework -
- 5. Shape From Silhouettes Reconstruction 5
- 6. Proposed Approach Updated Camera Poses Bounding Volume Estimation Volumetric Reconstruction RECONSTRUCTION Silhouette Extraction Global Transformation SLAM Point clouds & Camera Poses Free Viewpoint Animation Visualization Application SYNCHRONOUS DATA CAPTURE c1 c2 cC Storage Router 6
- 8. Silhouette Extraction Input Image Mask- RCNN Class (e.g. Person) Bounding Box(ROI) (m x m Mask) Detection (Mask-RCNN) Threshold 8 Annotation FC-CRF Silhouette Segmentation (FC-CRF) Updated Camera Poses Bounding Volume Estimation Volumetric Reconstruction RECONSTRUCTION Silhouette Extraction Global Transformation SLAM Point clouds & Camera Poses Free Viewpoint Animation Visualization Application SYNCHRONOUS DATA CAPTURE c1 c2 cC Stora ge Rout er
- 9. We get… Image Mask Overestimation Underestimation Annotation Silhouette 9
- 10. A Closer Look… 10 Input Estimation Ground Truth
- 11. Camera Calibration Parameters Global Transformation (Alignment & Anchor) Update Camera Pose [R|t]1 [R|t] 2 [R|t] c pose1 pose2 posec Unaligned Point Clouds Aligned Point Clouds Rigid Transformation Matrix Globally Updated Poses Point Cloud1 (reference) Point Cloud2 Point Cloudc 11 Updated Camera Poses Bounding Volume Estimation Volumetric Reconstruction RECONSTRUCTION Silhouette Extraction Global Transformation SLAM Point clouds & Camera Poses Free Viewpoint Animation Visualization Application SYNCHRONOUS DATA CAPTURE c1 c2 cC Stora ge Rout er
- 12. Bounding Volume Estimation Pmin(xmin, ymin, zmin) Pmax(xmax, ymax, zmax) X(X,Y,Z) x(u,v) x’(u’,v’) e e’ c c’ 12 Updated Camera Poses Bounding Volume Estimation Volumetric Reconstruction RECONSTRUCTION Silhouette Extraction Global Transformation SLAM Point clouds & Camera Poses Free Viewpoint Animation Visualization Application SYNCHRONOUS DATA CAPTURE c1 c2 cC Stora ge Rout er
- 13. Reconstruction Visual Hull (Shape from silhouettes) 13 Updated Camera Poses Bounding Volume Estimation Volumetric Reconstruction RECONSTRUCTION Silhouette Extraction Global Transformation SLAM Point clouds & Camera Poses Free Viewpoint Animation Visualization Application SYNCHRONOUS DATA CAPTURE c1 c2 cC Stora ge Rout er
- 14. Reconstruction Marching Cubes 14 Marching Cubes Voxel Representation Triangle Mesh
- 16. 4DFY Dataset 16 Hardware Capture Environment Non-static viewpoints Cluttered Background Occlusion Uncontrolled Shadow Illumination Change Reflective Surface
- 17. 4DFY Videos 17
- 18. 18
- 19. 19
- 20. Conclusion • Commodity hardware based dynamic object reconstruction framework • A novel technique for silhouette extraction • An effective method for estimating bounding volume • 4DFY Dataset 20
- 21. Future Work • Texture • Parallel Processing • GPU Acceleration • Improve Segmentation • Pose Estimation And Deform According To Pose • Key-frame Shape Interpolation 21
- 22. Thank You! Thank God for this dude’s dynamic reconstruction system! Gonna save us a lot of money and time. You can ask him some questions now. 22
- 23. 23
- 27. Visualization Application 27 Sort obj sequence Has objnext ? obj = obj0 1.Deactivate(obj) 2. obj = objnext Object Scale Render(obj) Object ScaleObject Scale Free Camera 360 Camera Free Viewpoint Animation Visualization Application Storage for Reconstructed object files yes no
- 28. Silhouette Extraction Performance 28 Table: Evaluation scores for silhouette extraction Table: Silhouette extraction computation time