Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2023 Jan-Dec:27:23312165231209913.
doi: 10.1177/23312165231209913.

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

Chengshi Zheng  1   2 , Huiyong Zhang  1   2 , Wenzhe Liu  1   2 , Xiaoxue Luo  1   2 , Andong Li  1   2 , Xiaodong Li  1   2 , Brian C J Moore  3
Affiliations
Review

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

Chengshi Zheng et al. Trends Hear. 2023 Jan-Dec.

Abstract

Frequency-domain monaural speech enhancement has been extensively studied for over 60 years, and a great number of methods have been proposed and applied to many devices. In the last decade, monaural speech enhancement has made tremendous progress with the advent and development of deep learning, and performance using such methods has been greatly improved relative to traditional methods. This survey paper first provides a comprehensive overview of traditional and deep-learning methods for monaural speech enhancement in the frequency domain. The fundamental assumptions of each approach are then summarized and analyzed to clarify their limitations and advantages. A comprehensive evaluation of some typical methods was conducted using the WSJ + Deep Noise Suppression (DNS) challenge and Voice Bank + DEMAND datasets to give an intuitive and unified comparison. The benefits of monaural speech enhancement methods using objective metrics relevant for normal-hearing and hearing-impaired listeners were evaluated. The objective test results showed that compression of the input features was important for simulated normal-hearing listeners but not for simulated hearing-impaired listeners. Potential future research and development topics in monaural speech enhancement are suggested.

Keywords: deep complex network; multistage learning; noise estimation; speech dereverberation; speech enhancement.

PubMed Disclaimer

Conflict of interest statement

Declaration of Conflicting InterestsThe authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1.
Figure 1.
A flow-process diagram of traditional methods.
Figure 2.
Figure 2.
Time-domain clean speech and noisy speech, and their corresponding magnitude and phase spectrograms. ( a1 ) time-domain clean speech; ( b1 ) magnitude spectrogram of clean speech with frame length 320 and frame shift 160; ( c1 ) phase spectrogram of clean speech with the same frame length and shift as ( b1 ); ( d1 ) magnitude spectrogram of clean speech with frame length 64 and frame shift 16; ( e1 ) phase spectrogram of clean speech with the same frame length and shift as ( d1 ). ( a2 ) to ( e2 ) correspond to ( a1 ) to ( e1 ) with noisy speech at SNR = 5 dB.
Figure 3.
Figure 3.
A generic flow-process diagram of deep learning methods.
Figure 4.
Figure 4.
Standard audiograms for two of the audiograms defined by Bisgaard et al. (2010): Norm: normal, N2: mild sensorineural loss, N3: moderate sensorineural loss.
See this image and copyright information in PMC

References

    1. Abdel-Hamid O., Mohamed Ar., Jiang H., Deng L., Penn G., Yu D. (2014). Convolutional neural networks for speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(10), 1533–1545. 10.1109/TASLP.2014.2339736 - DOI
    1. Aharon M., Elad M., Bruckstein A. (2006). K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54(11), 4311–4322. 10.1109/TSP.2006.881199 - DOI
    1. Allen J. (1977). Short term spectral analysis, synthesis, and modification by discrete Fourier transform. IEEE Transactions on Acoustics, Speech, and Signal Processing, 25(3), 235–238.
    1. Bando Y., Mimura M., Itoyama K., Yoshii K., Kawahara T. (2018). Statistical speech enhancement based on probabilistic integration of variational autoencoder and non-negative matrix factorization. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 716–720). IEEE.
    1. Benesty J., Chen J. (2011). Optimal time-domain noise reduction filters: A theoretical study. Springer. 10.1007/978-3-642-19601-0. - DOI

Publication types

LinkOut - more resources