Speech Enhancement Using Emotion-Dependent Codebooks

Abstract:
Several speech enhancement approaches utilize trained models of clean speech data, such as codebooks, Gaussian mixtures, and hidden Markov models. These models are typically trained on neutral clean speech data, without any emotion. However, in practical scenarios, emotional speech is a common occurrence, which brings into question the suitability of using models trained on neutral speech for enhancement of noisy emotional speech. We investigate this problem using the example of a codebook-based speech enhancement approach, which utilizes trained codebooks of linear prediction parameters. Anger and happiness are used as examples of emotions. Our experiments demonstrate that employing emotion-dependent speech codebooks results in a significant benefit over using emotion-independent codebooks for enhancing emotional noisy speech. We also present results using a Bayesian framework employing both emotiondependent and independent speech codebooks that exhibits a robust behavior when the type of emotion is not known a priori. Index Terms — Speech enhancement, codebook, emotional speech