Context (archaeology)

Linguistics

Philosophy

Scene graph

Biology

Statistics

Theoretical computer science

Operating system

Set (abstract data type)

Computer science

Feature matching

Code (set theory)

Paleontology

Rendering (computer graphics)

Graph

Pattern recognition (psychology)

Mathematics

Feature (linguistics)

Differentiable function

Mathematical analysis

Artificial intelligence

Matching (statistics)

Programming language

Artificial neural network

Feature extraction

Machine learning

Heuristics

This paper introduces SuperGlue, a neural network that matches two sets of local features by jointly finding correspondences and rejecting non-matchable points. Assignments are estimated by solving a differentiable optimal transport problem, whose costs are predicted by a graph neural network. We introduce a flexible context aggregation mechanism based on attention, enabling SuperGlue to reason about the underlying 3D scene and feature assignments jointly. Compared to traditional, hand-designed heuristics, our technique learns priors over geometric transformations and regularities of the 3D world through end-to-end training from image pairs. SuperGlue outperforms other learned approaches and achieves state-of-the-art results on the task of pose estimation in challenging real-world indoor and outdoor environments. The proposed method performs matching in real-time on a modern GPU and can be readily integrated into modern SfM or SLAM systems. The code and trained weights are publicly available at github.com/magicleap/SuperGluePretrainedNetwork.

SuperGlue: Learning Feature Matching With Graph Neural Networks