Deep learning

Pattern recognition (psychology)

Test set

Process engineering

Feature (linguistics)

Mathematics

Domain (mathematical analysis)

Gene

Geodesy

Extractor

Machine learning

Engineering

Mathematical analysis

Philosophy

Workflow

Benchmark (surveying)

Artificial intelligence

Geography

Generalization

Chemistry

Computer science

Linguistics

Feature extraction

Deep neural networks

Artificial neural network

Training set

Biochemistry

Robustness (evolution)

Database

Classifier (UML)

Domain generalization (DG) is the challenging and topical problem of learning models that generalize to novel testing domains with different statistics than a set of known training domains. The simple approach of aggregating data from all source domains and training a single deep neural network end-to-end on all the data provides a surprisingly strong baseline that surpasses many prior published methods. In this paper we build on this strong baseline by designing an episodic training procedure that trains a single deep network in a way that exposes it to the domain shift that characterises a novel domain at runtime. Specifically, we decompose a deep network into feature extractor and classifier components, and then train each component by simulating it interacting with a partner who is badly tuned for the current domain. This makes both components more robust, ultimately leading to our networks producing state-of-the-art performance on three DG benchmarks. Furthermore, we consider the pervasive workflow of using an ImageNet trained CNN as a fixed feature extractor for downstream recognition tasks. Using the Visual Decathlon benchmark, we demonstrate that our episodic-DG training improves the performance of such a general purpose feature extractor by explicitly training a feature for robustness to novel problems. This shows that DG training can benefit standard practice in computer vision.

Episodic Training for Domain Generalization