Separation (statistics)

Microphone array

Telecommunications

Source separation

Computer science

Sound pressure

Acoustics

Beamforming

Physics

Mathematics

Impulse response

Block (permutation group theory)

Mathematical analysis

Artificial intelligence

Geometry

Artificial neural network

Microphone

Machine learning

Speech recognition

IEEE/ACM Transactions on Audio Speech and Language Processing

We propose multi-microphone complex spectral mapping, a simple way of applying deep learning for time-varying non-linear beamforming, for speaker separation in reverberant conditions. We aim at both speaker separation and dereverberation. Our study first investigates offline utterance-wise speaker separation and then extends to block-online continuous speech separation (CSS). Assuming a fixed array geometry between training and testing, we train deep neural networks (DNN) to predict the real and imaginary (RI) components of target speech at a reference microphone from the RI components of multiple microphones. We then integrate multi-microphone complex spectral mapping with minimum variance distortionless response (MVDR) beamforming and post-filtering to further improve separation, and combine it with frame-level speaker counting for block-online CSS. Although our system is trained on simulated room impulse responses (RIR) based on a fixed number of microphones arranged in a given geometry, it generalizes well to a real array with the same geometry. State-of-the-art separation performance is obtained on the simulated two-talker SMS-WSJ corpus and the real-recorded LibriCSS dataset.

Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation