Filter bank

Filter (signal processing)

Paleontology

Artificial intelligence

Acoustic model

Exploit

Political science

Labeled data

Representation (politics)

Context (archaeology)

Speech recognition

Computer vision

Computer security

Computer science

Speech processing

Biology

Politics

We propose a novel approach to semi-supervised automatic speech recognition (ASR). We first exploit a large amount of unlabeled audio data via representation learning, where we reconstruct a temporal slice of filterbank features from past and future context frames. The resulting deep contextualized acoustic representations (DeCoAR) are then used to train a CTC-based end-to-end ASR system using a smaller amount of labeled audio data. In our experiments, we show that systems trained on DeCoAR consistently outperform ones trained on conventional filterbank features, giving 42% and 19% relative improvement over the baseline on WSJ eval92 and LibriSpeech test-clean, respectively. Our approach can drastically reduce the amount of labeled data required; unsupervised training on LibriSpeech then supervision with 100 hours of labeled data achieves performance on par with training on all 960 hours directly. Pre-trained models and code will be released online.

Deep Contextualized Acoustic Representations for Semi-Supervised Speech Recognition