6D Pose Estimation via Keypoint Heatmap Regression with RGB-D Residual Neural Networks
Ismail Aljosevic, Amir Masoud Almasi, Ana Parovic, Ashkan Shafiei
TLDR
This paper proposes a 6D pose estimation framework using keypoint heatmap regression, achieving high accuracy with RGB-D fusion.
Key contributions
- Introduces a modular 6D pose estimation framework combining YOLOv10m for detection and ResNet18 for keypoint heatmaps.
- Estimates 6D object pose using PnP RANSAC from 2D keypoints extracted from predicted heatmaps.
- Incorporates depth data via a cross-fusion architecture, significantly boosting accuracy over RGB-only methods.
- Achieves 92.41% ADD-based accuracy on LINEMOD with RGB-D fusion, outperforming RGB-only models.
Why it matters
This paper offers a robust, modular 6D pose estimation framework crucial for robotics and AR. Its significant accuracy boost from RGB-D fusion via a cross-fusion architecture provides a promising direction. The strong results and modular design make it valuable for practical applications.
Original Abstract
In this paper, we propose a modular framework for 6D pose estimation based on keypoint heatmap regression. Our approach combines YOLOv10m for object detection with a ResNet18-based network that predicts 2D heatmaps from RGB images. Keypoints extracted from these heatmaps are used to estimate the 6D object pose via the PnP RANSAC algorithm. We compare different keypoint selection strategies to assess their impact on pose accuracy. Additionally, we extend the baseline by incorporating depth data using a cross-fusion architecture, which enables interaction between RGB and depth features at multiple stages. We further explore general training improvements, such as experimenting with activation functions and learning rate scheduling strategies to improve model performance. Our best RGB-only model achieved a mean ADD-based accuracy of 84.50%, while the RGB-D fusion model reached 92.41% on the LINEMOD dataset. The code is available at https://github.com/ameermasood/HeatNet.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.