Deep learning

Physics

Light field

Wireless

Optics

Light sheet fluorescence microscopy

Iterative reconstruction

Microscopy

Artificial intelligence

Throughput

Scanning confocal electron microscopy

Computer science

Computer vision

Artificial neural network

Telecommunications

Convolutional neural network

Nature Methods

Visualizing dynamic processes over large, three-dimensional fields of view at high speed is essential for many applications in the life sciences. Light-field microscopy (LFM) has emerged as a tool for fast volumetric image acquisition, but its effective throughput and widespread use in biology has been hampered by a computationally demanding and artifact-prone image reconstruction process. Here, we present a framework for artificial intelligence–enhanced microscopy, integrating a hybrid light-field light-sheet microscope and deep learning–based volume reconstruction. In our approach, concomitantly acquired, high-resolution two-dimensional light-sheet images continuously serve as training data and validation for the convolutional neural network reconstructing the raw LFM data during extended volumetric time-lapse imaging experiments. Our network delivers high-quality three-dimensional reconstructions at video-rate throughput, which can be further refined based on the high-resolution light-sheet images. We demonstrate the capabilities of our approach by imaging medaka heart dynamics and zebrafish neural activity with volumetric imaging rates up to 100 Hz. A deep learning–based algorithm enables efficient reconstruction of light-field microscopy data at video rate. In addition, concurrently acquired light-sheet microscopy data provide ground truth data for training, validation and refinement of the algorithm.

Deep learning-enhanced light-field imaging with continuous validation