Invariant (physics)

Political science

Mathematics

Mathematical physics

Politics

Computer science

Counterexample

Machine learning

Generalization

Representation (politics)

Classifier (UML)

Algorithm

Theoretical computer science

Domain adaptation

Domain (mathematical analysis)

Artificial intelligence

Mathematical analysis

Discrete mathematics

Due to the ability of deep neural nets to learn rich representations, recent advances in unsupervised domain adaptation have focused on learning domain-invariant features that achieve a small error on the source domain. The hope is that the learnt representation, together with the hypothesis learnt from the source domain, can generalize to the target domain. In this paper, we first construct a simple counterexample showing that, contrary to common belief, the above conditions are not sufficient to guarantee successful domain adaptation. In particular, the counterexample exhibits conditional shift: the class-conditional distributions of input features change between source and target domains. To give a sufficient condition for domain adaptation, we propose a natural and interpretable generalization upper bound that explicitly takes into account the aforementioned shift. Moreover, we shed new light on the problem by proving an information-theoretic lower bound on the joint error of any domain adaptation method ...

On learning invariant representations for domain adaptation