Paper Info
Title
PROSODY AND VOICE FACTORIZATION FOR FEW-SHOT SPEAKER ADAPTATION IN THE CHALLENGE M2VOC 2021
Abstract
The paper describes the CASIA speech synthesis system entry for challenge M2VoC 2021. The low similarity and naturalness of synthesized speech remains a challenging problem for speaker adaptation with few resources. Since the end-to-end acoustic model is too complex to interpret, overfitting will occur when training with few data. To prevent the model from overfitting, this paper proposes a novel speaker adaptation framework that decomposes the prosody and voice characteristics in the end-to-end model. A prosody control attention is proposed to control the phonemes' duration of different speakers. To make the attention controlled by the prosody information, a set of phoneme-level transition tokens is auto-learned from the prosody encoder in our framework and these transition tokens can determine the duration of phonemes in the attention mechanism. Secondly, when we need to use small data set for speaker adaptation, we just need to adapt the speaker related prosody model and decoder, which can prevent the model from overfitting. Further, we use a data puring model to automatically optimize the quality of datasets. Experiments demonstrate the effectiveness of speaker adaptation based on our method, and we (team identifier is T03) get the top three results in competition M2VoC by using this framework.
Year
DOI
Venue
2021
10.1109/ICASSP39728.2021.9414427
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021)
Keywords
DocType
Citations 
speech synthesis, end-to-end model, prosody and voice factorization, few-shot speaker adaptation, the M2VoC challenge
Conference
0
PageRank 
References 
Authors
0.34
0
7
Name
Order
Citations
PageRank
Tao Wang102.70
Ruibo Fu215.11
Jiangyan Yi31917.99
Jianhua Tao4848138.00
Zhengqi Wen58624.41
Chunyu Qiang602.03
Shiming Wang700.68