Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks
•
0 likes•231 views
Conference paper presentation at IEEE/ASME International Conference on Advanced Intelligent Mechatronics
Recommended
Recommended
More Related Content
Similar to Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks
Similar to Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks (11)
More from Justas Miseikis
More from Justas Miseikis (7)
Recently uploaded
Recently uploaded (20)
Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks
- 1. Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded CNNs Justinas Mišeikis, Patrick Knöbelreiter, Inka Brijacak, Saeed Yahyanejad, Kyrre Glette, Ole Jakob Elle and Jim Torresen
- 7. Can we solve it? Machine Learning
- 8. Input Image Intermediate: Robot Mask Robot 3D Joint Coordinate Estimation
- 9. Input Image Intermediate: Robot Mask Robot 3D Joint Coordinate Estimation 2D Color Image Input
- 10. Input Image Intermediate: Robot Mask Robot 3D Joint Coordinate Estimation Outputs
- 11. First Step: Data Collection Datasets of the whole range of Universal Robots were collected: UR3, UR5 and UR10
- 12. Cascaded CNN Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Conv 32F FC CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output Conv 32F Filter Size: 3x3 Conv 64F Conv 128F Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2
- 13. Cascaded CNN Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Conv 32F FC CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output Conv 32F Filter Size: 3x3 Conv 64F Conv 128F Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Classification
- 14. Cascaded CNN Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Conv 32F FC CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output Conv 32F Filter Size: 3x3 Conv 64F Conv 128F Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Overlay
- 15. Cascaded CNN Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Conv 32F FC CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output Conv 32F Filter Size: 3x3 Conv 64F Conv 128F Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Regression
- 17. Training Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 Conv 32F Filter Size: 3x3 Dilation 2x2 Input Output Input Conv 32F Filter Size: 3x3 Dilation: 2x2 FC 500 CNN for Robot Body Classification CNN for Robot Joint Coordinates Estimation Output 21 Conv 32F Filter Size: 3x3 Conv 64F Filter Size: 3x3 Dilation: 5x5 Conv 128F Filter Size: 2x2 Dilation: 3x3 Conv 32F Filter Size: 3x3 Dilation 3x3 Conv 32F Filter Size: 3x3 Dilation 5x5 Conv 32F Filter Size: 3x3 Dilation 3x3 Max Pooling 2x2 Max Pooling 2x2 The Whole System Each CNN Separately
- 18. Results UR3 UR5 UR10 Mask Accuracy, % 93.1% 98.1% 92.8% Coordinates Error (Separate) 2.5 cm 2.02 cm 3.21 cm Coordinates Error (Full System) 2.57 cm 2.42 cm 3.89 cm Number of Training Samples 926 2520 904
- 21. More Application Areas Photo Source: https://www.grasp.upenn.edu/projects/haptics-human-robot-interaction Photo Source: https://www.youtube.com/watch?v=6VOvi19mFfE Photo Source: https://www.raspberrypi.org/blog/wearable-pi-zero-camera/
- 22. Future Work ‣ Expand the method to more robot types: transfer learning ‣ Try multi-objective CNN for a single-training system ‣ Test the method in a real-world scenario and human-robot interaction tasks
- 23. Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded CNNs Justinas Mišeikis, Patrick Knöbelreiter, Inka Brijacak, Saeed Yahyanejad, Kyrre Glette, Ole Jakob Elle and Jim Torresen Thank You!