Depth-Based Real Time Head Motion Tracking Using 3D Template Matching
•Download as PPTX, PDF•
1 like•1,279 views
In this work, we propose a system to estimate head poses only using depth information in real-time. An optimization method based on 3D model fitting is developed. We iteratively minimize the distance between source and target point clouds of a user’s head. The method give fully real-time responses (30fps) without the GPU speedup. We adopt a commodity depth sensor named Microsoft Kinect as well as Asus Xtion, and use the depth image as the only input so that our system will not be affected by illumination variations. However, the simplicity of this acquisition device comes at the cost of frequent noises in the acquired data. We demonstrate that 6 degrees of freedom real-time head motion tracking in 3D space can be achieved with such noisy depth data.
Recommended
Recommended
More Related Content
Similar to Depth-Based Real Time Head Motion Tracking Using 3D Template Matching
Similar to Depth-Based Real Time Head Motion Tracking Using 3D Template Matching (20)
Recently uploaded
Recently uploaded (20)
Depth-Based Real Time Head Motion Tracking Using 3D Template Matching
- 1. Depth-Based Real Time Head Motion Tracking Using 3D Template Matching 以深度影像及三維樣板比對為基礎之即時頭部 運動軌跡追蹤 學生：屠愚 指導教授：歐陽明博士 1
- 2. Outline • Introduction • System Overview • Algorithm • Results & System Demo • Conclusion & Future Work 2
- 4. What’s Head Motion Tracking 4 Y-axis Yaw Pitch Roll Y translation Real-time reconstruction of this 6-DoF motion vector given a stream of video input.
- 5. 5 Why Head Motion Tracking GVAainzdDeiem oOa anGrtciaieionmngtea tion
- 6. Related Work • Color Image Based Methods “Head Pose Estimation in Computer Vision : A Survey”, E. Murphy-Chutorian and M. M. Trivedi., PAMI 09 • Appearance Template Methods • Feature Tracking Methods • Detector Arrays • Nonlinear Regression Methods • Manifold Embedding Methods • Flexible Models • Geometric Methods • Hybrid Methods • Too sensitive to illumination variations! 6
- 7. • SwissRanger SR4000 – Messa Image (2006) – $9000 • CameCube 2.0 – PMD Technology (2002) – $12000 7 Depth Camera
- 8. • Kinect by Microsoft – $149.99 • Xtion Pro by Asus – $189 . – $300 . 8 Depth Camera
- 9. 9 Related Work “Real-time performance-based facial animation” T. Weise et al., SIGGRAPH 2011 “Real Time Head Pose Estimation with Random Regression Forests”, G. Fanelli et al., CVPR 2011
- 11. 11
- 12. 12
- 13. Flow Chart Least Square Error Method Inverse Rotation Nose Detection Sampling Iterative Optimization Method User Acting Avatar Control 13 Depth Data Acquisition Real-time Head Pose Estimation
- 14. Algorithm Method 1 :Least square error method 14
- 16. Z-axis 16 Nose Detection Zmin Zmin+5
- 20. Nose Tracking Flow Chart 20 Nose Detection Nose푡 푥, 푦 Nose푡−1 푥, 푦 , 휃푡−1, 휓푡−1, 휙푡−1, Depth Map푡 Searching Window Setting Inverse Rotation z-min Searching
- 21. Least Square Plane Fitting 21 Yaw & Pitch ! Ax+By+Cz=1
- 22. Least Square Ellipse Fitting 23 Roll !
- 23. Dynamic weighted average filter 0.01 0.1 1 10 퐻 ∙ 푚푎푥푙∈[1,푘] 푡푖 − 푡푖−푙 ∗ = 푘 푤푗 푡푖−푗 푗=0 푘 푤푗 “Real-time performance-based facial animation” T. Weise et al., Siggraph 2011 24 Flickering Issue 푛표푟푚푎푙푖푧푒푑 푤푒푖푔ℎ푡 휔 푓푟푎푚푒 푖 푖-1 푖-2 푖-3 푖-4 푤푗 = 푒−푡 푗∙퐻∙푚푎푥푙∈[1,푘] 푡푖−푡푖−푙 푖 푗=0
- 24. Algorithm Method 2 :Optimization method 25
- 25. 26 User Acting Flow Chart Least Square Error Method Inverse Rotation Nose Detection Sampling Depth Data Iterative Optimization Method Acquisition Real-time Head Pose Estimation Avatar Control
- 26. 27 Model Point Cloud
- 27. 28 Sample Point Cloud
- 28. • 푚표푡푖표푛 푣푒푐푡표푟 푎 Θ, 휓, 휙, 푡푥 , 푡푦, 푡푧 푝∈푃푆 min(푑 푝′, 푃푀 ) • 퐸 푎, 푃푆, 푃푀 = 푝 • 푅푒푠푢푙푡 = 푎∗ = arg 푚푖푛 (퐸) 29 Energy Function
- 29. 30 Optimization • Gradient Decent Algorithm 푎푘+1 = 푎푘 − 훼푘훻퐸(푎푘 ) Non-differentiable • 12 Possible Gradients 훻1 = 1 0 0 0 0 0 , 훻2 = −1 0 0 0 0 0 , 훻3 = 0 1 0 0 0 0 , 훻4 = 0 −1 0 0 0 0 , 훻5 = 0 0 1 0 0 0 , 훻6 = 0 0 −1 0 0 0 , 훻7 = 0 0 0 1 0 0 , 훻8 = 0 0 0 −1 0 0 , 훻9 = 0 0 0 0 1 0 , 훻10 = 0 0 0 0 −1 0 , 훻11 = 0 0 0 0 0 1 , 훻12 = 0 0 0 0 0 −1
- 30. 31 Optimization
- 31. Results Of Method 1 32
- 32. 33
- 33. 34
- 34. 35
- 35. Results Of Method 2 36
- 36. 37
- 37. 38
- 38. 39
- 39. Our Method with 50% Database (Without Bad Initialization) 40 Our Method with Whole Database Our Method with 70% Database (Without Crash)
- 40. 41
- 41. System Demo 42
- 42. Conclusion 43
- 43. • Contribution: – Two novel algorithms. – Not affected by varying lighting conditions. – Real-time responses without GPU acceleration. – Outperforms the state-of-the-art approach (G. Fanelli et al., CVPR 2011). • Future Work: – Tracking without nose. – Combine with NITE skeleton tracking. – Facial Expression Recognition / Retargeting. 44
- 44. Thank You Very Much 45
Editor's Notes
- Messa Image 2006, Swiss Ranger 9000
- 3DV 2000 release zcam, 2009/3 brought by M$