Motion (physics)

Quantum mechanics

Process (computing)

Event (particle physics)

Image processing

Operating system

Telecommunications

Image restoration

Computer science

Physics

Motion estimation

Image (mathematics)

Motion blur

Artificial intelligence

Deep learning

Deblurring

Convolutional neural network

Computer vision

Frame (networking)

Recovering sharp video sequence from a motion-blurred image is highly ill-posed due to the significant loss of motion information in the blurring process. For event-based cameras, however, fast motion can be captured as events at high frame rate, raising new opportunities to exploring effective solutions. In this paper, we start from a sequential formulation of event-based motion deblurring, then show how its optimization can be unfolded with a novel end-toend deep architecture. The proposed architecture is a convolutional recurrent neural network that integrates visual and temporal knowledge of both global and local scales in principled manner. To further improve the reconstruction, we propose a differentiable directional event filtering module to effectively extract rich boundary prior from the evolution of events. We conduct extensive experiments on the synthetic GoPro dataset and a large newly introduced dataset captured by a DAVIS240C camera. The proposed approach achieves state-of-the-art reconstruction quality, and generalizes better to handling real-world motion blur.

Learning Event-Based Motion Deblurring