Messenger RNA

Random variable

Control engineering

Independent and identically distributed random variables

Translation (biology)

Pattern recognition (psychology)

Organic chemistry

Mathematics

Feed forward

Contextual image classification

Gene

Algorithm

Image (mathematics)

Statistics

Machine learning

Engineering

Residual

Artificial intelligence

Network architecture

Set (abstract data type)

Chemistry

Programming language

Code (set theory)

Layer (electronics)

Image translation

Computer science

Computer security

Artificial neural network

Biochemistry

Perceptron

IEEE Transactions on Pattern Analysis and Machine Intelligence

We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification. It is a simple residual network that alternates (i) a linear layer in which image patches interact, independently and identically across channels, and (ii) a two-layer feed-forward network in which channels interact independently per patch. When trained with a modern training strategy using heavy data-augmentation and optionally distillation, it attains surprisingly good accuracy/complexity trade-offs on ImageNet. We also train ResMLP models in a self-supervised setup, to further remove priors from employing a labelled dataset. Finally, by adapting our model to machine translation we achieve surprisingly good results. We share pre-trained models and our code based on the Timm library.

ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training