标题： QS-TTS：通过向量量化自监督语音表示学习实现半监督文本到语音合成 显示英文标题

标题： QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning

摘要： 本文提出了一种新颖的半监督TTS框架QS-TTS，通过利用更多未标记语音音频的向量量化自监督语音表示学习（VQ-S3RL），在较低监督数据需求的情况下提高TTS质量。该框架包含两个VQ-S3R学习器：首先，主学习器旨在通过MSMC-VQ-GAN结合对比S3RL提供生成式多阶段多码本（MSMC）VQ-S3R，并将其解码为高质量音频；然后，关联学习器通过VQ-VAE进一步将MSMC表示抽象为高度紧凑的VQ表示。这两个生成式VQ-S3R学习器为TTS提供了有益的语音表示和预训练模型，在较低的监督数据需求下显著提高了合成质量。QS-TTS在实验中通过主观和客观测试在各种场景下进行了全面评估。结果有力地证明了QS-TTS的优越性能，在监督或半监督基线TTS方法中获得了最高的MOS，尤其是在低资源场景中。此外，比较TTS中的各种语音表示和迁移学习方法进一步验证了所提出的VQ-S3RL对TTS的显著改进，显示出最佳的音频质量和可懂度指标。随着监督数据的减少，QS-TTS的合成质量衰减较慢的趋势进一步突显了其对监督数据的较低需求，表明其在低资源场景中具有巨大的潜力。 显示英文摘要

摘要： This paper proposes a novel semi-supervised TTS framework, QS-TTS, to improve TTS quality with lower supervised data requirements via Vector-Quantized Self-Supervised Speech Representation Learning (VQ-S3RL) utilizing more unlabeled speech audio. This framework comprises two VQ-S3R learners: first, the principal learner aims to provide a generative Multi-Stage Multi-Codebook (MSMC) VQ-S3R via the MSMC-VQ-GAN combined with the contrastive S3RL, while decoding it back to the high-quality audio; then, the associate learner further abstracts the MSMC representation into a highly-compact VQ representation through a VQ-VAE. These two generative VQ-S3R learners provide profitable speech representations and pre-trained models for TTS, significantly improving synthesis quality with the lower requirement for supervised data. QS-TTS is evaluated comprehensively under various scenarios via subjective and objective tests in experiments. The results powerfully demonstrate the superior performance of QS-TTS, winning the highest MOS over supervised or semi-supervised baseline TTS approaches, especially in low-resource scenarios. Moreover, comparing various speech representations and transfer learning methods in TTS further validates the notable improvement of the proposed VQ-S3RL to TTS, showing the best audio quality and intelligibility metrics. The trend of slower decay in the synthesis quality of QS-TTS with decreasing supervised data further highlights its lower requirements for supervised data, indicating its great potential in low-resource scenarios.