Function (biology)

Algorithm

Residual

Linguistics

Engineering

Philosophy

Biology

Systems engineering

Operating system

Set (abstract data type)

Computer science

Code (set theory)

Evolutionary biology

Fusion

Task (project management)

Pattern recognition (psychology)

Image (mathematics)

Image fusion

Feature (linguistics)

Artificial intelligence

End-to-end principle

Programming language

Deep learning

Machine learning

Encoder

Computer vision

Information Fusion

In the image fusion field, the design of deep learning-based fusion methods is far from routine. It is invariably fusion-task specific and requires a careful consideration. The most difficult part of the design is to choose an appropriate strategy to generate the fused image for a specific task in hand. Thus, devising learnable fusion strategy is a very challenging problem in the community of image fusion. To address this problem, a novel end-to-end fusion network architecture (RFN-Nest) is developed for infrared and visible image fusion. We propose a residual fusion network (RFN) which is based on a residual architecture to replace the traditional fusion approach. A novel detail-preserving loss function, and a feature enhancing loss function are proposed to train RFN. The fusion model learning is accomplished by a novel two-stage training strategy. In the first stage, we train an auto-encoder based on an innovative nest connection (Nest) concept. Next, the RFN is trained using the proposed loss functions. The experimental results on public domain data sets show that, compared with the existing methods, our end-to-end fusion network delivers a better performance than the state-of-the-art methods in both subjective and objective evaluation. The code of our fusion method is available at https://github.com/hli1221/imagefusion-rfn-nest . • Residual fusion network (RFN) is proposed to supersede handcrafted fusion strategies. • Two-stage training strategy is developed to train RFN. • Detail preservation and feature enhancement loss functions are designed. • The proposed method achieves better performance compare with existing methods.

RFN-Nest: An end-to-end residual fusion network for infrared and visible images