Geodesy

Benchmark (surveying)

Philosophy

Distillation

Statistics

Embedded system

Theoretical computer science

Operating system

Anomaly detection

Generalizability theory

Computer science

Simple (philosophy)

Anomaly (physics)

Embedding

Physics

Organic chemistry

Mathematics

Politics

Geography

Political science

Epistemology

Bottleneck

Condensed matter physics

Artificial intelligence

Representation (politics)

Chemistry

Class (philosophy)

Machine learning

Encoder

Heuristics

Knowledge distillation (KD) achieves promising results on the challenging problem of unsupervised anomaly detection (AD). The representation discrepancy of anomalies in the teacher-student (T-S) model provides essential evidence for AD. However, using similar or identical architectures to build the teacher and student models in previous studies hinders the diversity of anomalous representations. To tackle this problem, we propose a novel T-S model consisting of a teacher encoder and a student decoder and introduce a simple yet effective “reverse distillation” paradigm accordingly. Instead of receiving raw images directly, the student network takes teacher model's one-class embedding as input and targets to restore the teacher's multi-scale representations. Inherently, knowledge distillation in this study starts from abstract, high-level presentations to low-level features. In addition, we introduce a trainable one-class bottleneck embedding (OCBE) module in our T-S model. The obtained compact embedding effectively preserves essential information on normal patterns, but aban-dons anomaly perturbations. Extensive experimentation on AD and one-class novelty detection benchmarks shows that our method surpasses SOTA performance, demonstrating our proposed approach's effectiveness and generalizability.

Anomaly Detection via Reverse Distillation from One-Class Embedding