Pose

Engineering

Aerospace engineering

NASA Deep Space Network

Test set

Ambiguity

Computer science

Rendering (computer graphics)

Artificial intelligence

Space debris

Programming language

Spacecraft

Deep learning

Rendezvous

Computer vision

On-orbit proximity operations in space rendezvous, docking and debris removal require precise and robust 6D pose estimation under a wide range of lighting conditions and against highly textured background, i.e., the Earth. This paper investigates leveraging deep learning and photorealistic rendering for monocular pose estimation of known uncooperative spacecraft. We first present a simulator built on Unreal Engine 4, named URSO, to generate labeled images of spacecraft orbiting the Earth, which can be used to train and evaluate neural networks. Secondly, we propose a deep learning framework for pose estimation based on orientation soft classification, which allows modelling orientation ambiguity as a mixture model. This framework was evaluated both on URSO datasets and the European Space Agency pose estimation challenge. In this competition, our best model achieved 3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rd</sup> place on the synthetic test set and 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> place on the real test set. Moreover, our results show the impact of several architectural and training aspects, and we demonstrate qualitatively how models learned on URSO datasets can perform on real images from space.

Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering