Slide for study session given by Ryosuke Sasaki at Arithmer inc. It is a summary of recent methods for object pose estimation in robotics using deep learning. He entered Ph.D course at Univ. of Tokyo in April 2020. Arithmer株式会社は東京大学大学院数理科学研究科発の数学の会社です。私達は現代数学を応用して、様々な分野のソリューションに、新しい高度AIシステムを導入しています。AIをいかに上手に使って仕事を効率化するか、そして人々の役に立つ結果を生み出すのか、それを考えるのが私たちの仕事です。 Arithmer began at the University of Tokyo Graduate School of Mathematical Sciences. Today, our research of modern mathematics and AI systems has the capability of providing solutions when dealing with tough complex issues. At Arithmer we believe it is our job to realize the functions of AI through improving work efficiency and producing more useful results for society.

1. Object Pose Estimation
Arithmer R3 Div. R3 - Ryosuke Sasaki

2. • Correspondence-based method : Find Correspondences between 2D points and 3D points (PnP, ICP)
• Template-based method: Extract the gradient information for matching, and refined(ICP)
• Voting-based method: Every local predicts a result, and refined using RANSAC
• Regression-based method: Represent pose suitable for CNN

3. Arxiv:1905.06658

4. Arxiv:1905.06658
Correspondence-based method
With rich textures to get image features

5. Arxiv:1905.06658
PnP solver
Estimate extrinsic camera parameter using known 3D points in a world coordinate
There are many many solvers (e.g. P3P, P5P.., using RANSAC or Eigen value decomposition)

6. ICP algorithm
Iterative matching for 2 point clouds
1. Correspondence nearest neighbor points
2. Translate a point cloud set to match the other sets
3. iteration above

7. Arxiv:1905.06658
ICP
Estimate extrinsic camera parameter using known 3D points in a world coordinate
There are many many solvers (e.g. P3P, P5P.., using RANSAC or Eigen value decomposition)

8. SegICP
Segmentation and depth image-based partial
Registration to obtain object’s pose
SegNet
Multi-hypothesis object pose:
Point-to-point ICP
1cm pose error and < 5°error

9. Arxiv:1905.06658
Template-based method
Using gradient information in RGB-D or RGB image
Without rich texture
ICP

10. Arxiv:1905.06658
Voting-based method
Objects with occlusions
Dense class labeling

11. Regression-based method
PoseCNN
- Localizing the center of objects
- Predicting its distance from camera
- Regressing rotation to a quarternion

12. Regression-based method
PoseCNN architecture
- Feature extraction
- Embedding
- Classification and regression
Two loss function for quaternion
- PoseLoss
- ShapeMatch-Loss

13. Regression-based method
DeepIM
Iterative refinement using DNN
Enable to combine other estimator

14. Regression-based method
DeepIM
Iterative refinement using DNN
Flownet: estimate optical flow

15. Regression-based method
CullNet
Single view image-based object pose estimation
Culling false positives
Calibrate the confidence score

16. Correspondence-based and Regression-based method
CullNet Architecture
Using Yolov3 for keypoints
Cullnet output confidence
of objects pose by PnP.

17. Voting-based and Regression-based method
DenseFusion
Estimating 6D pose of known objects

18. Comparison
ADD (Average Distance of Model Point)
ADD-S (for symmetric objects
Error smaller than threshold(< 2cm)
From the tables, we can see that DenseFusion
achieved the highest accuracy comparing with other
regression-based methods.
novel local feature fusion scheme using both RGB
and depth images

19. 19