Gene

Pose

Articulated body pose estimation

Messenger RNA

Robustness (evolution)

Operating system

Cardiology

Biochemistry

Computer science

Rotation (mathematics)

Translation (biology)

Occlusion

Pattern recognition (psychology)

Image (mathematics)

3D pose estimation

RGB color model

Artificial intelligence

Medicine

Chemistry

RANSAC

Encoder

Computer vision

Directly regressing all 6 degrees-of-freedom (6DoF) for the object pose (i.e. the 3D rotation and translation) in a cluttered environment from a single RGB image is a challenging problem. While end-to-end methods have recently demonstrated promising results at high efficiency, they are still inferior when compared with elaborate PnP/RANSAC-based approaches in terms of pose accuracy. In this work, we address this shortcoming by means of a novel reasoning about self-occlusion, in order to establish a two-layer representation for 3D objects which considerably enhances the accuracy of end-to-end 6D pose estimation. Our framework, named SO-Pose, takes a single RGB image as input and respectively generates 2D-3D correspondences as well as self-occlusion information harnessing a shared encoder and two separate decoders. Both outputs are then fused to directly regress the 6DoF pose parameters. Incorporating cross-layer consistencies that align correspondences, self-occlusion and 6D pose, we can further improve accuracy and robustness, surpassing or rivaling all other state-of-the-art approaches on various challenging datasets.

SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation