Yu, C; Li, Y; Zu, W; Sun, F; Tian, Z; Wang, J; (2023) Cross-utterance Conditioned Coherent Speech Editing. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. (pp. pp. 2108-2112). ISCA: Dublin, Ireland.

Preview

Text Wang_yu23d_interspeech.pdf Download (1MB) | Preview

Abstract

Text-based speech editing systems are developed to enable users to modify speech based on the transcript. Existing state-of-the-art editing systems based on neural networks do partial inferences with no exception, that is, only generate new words that need to be replaced or inserted. This manner usually leads to the prosody of the edited part being inconsistent with the surrounding speech and a failure to handle the alteration of intonation. To address these problems, we propose a cross-utterance conditioned coherent speech editing system, that first does the entire reasoning at the inference time. Our proposed system can generate speech by utilizing speaker information, context, acoustic features, and the mel-spectrogram from the original audio. Experiments conducted on subjective and objective metrics demonstrate that our approach outperforms the baseline on various editing operations regarding naturalness and prosody consistency.

Download activity - last month

Export as XMLCSVJSON

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item