Quantum mechanics

Engineering

Regularization (linguistics)

Constraint (computer-aided design)

Set (abstract data type)

Scalability

Computer science

Code (set theory)

Mechanical engineering

Physics

Database

Inference

Artificial intelligence

Programming language

Deep learning

Artificial neural network

Recurrent neural network

Volume (thermodynamics)

Computer vision

Deep learning has recently demonstrated its excellent performance for multi-view stereo (MVS). However, one major limitation of current learned MVS approaches is the scalability: the memory-consuming cost volume regularization makes the learned MVS hard to be applied to high-resolution scenes. In this paper, we introduce a scalable multi-view stereo framework based on the recurrent neural network. Instead of regularizing the entire 3D cost volume in one go, the proposed Recurrent Multi-view Stereo Network (R-MVSNet) sequentially regularizes the 2D cost maps along the depth direction via the gated recurrent unit (GRU). This reduces dramatically the memory consumption and makes high-resolution reconstruction feasible. We first show the state-of-the-art performance achieved by the proposed R-MVSNet on the recent MVS benchmarks. Then, we further demonstrate the scalability of the proposed method on several large-scale scenarios, where previous learned approaches often fail due to the memory constraint. Code is available at https://github.com/YoYo000/MVSNet.

Recurrent MVSNet for High-Resolution Multi-View Stereo Depth Inference