标题： USTED：通过统一的语音和文本编码器-解码器改进自动语音识别 显示英文标题

标题： USTED: Improving ASR with a Unified Speech and Text Encoder-Decoder

摘要： 通过结合外部文本数据来改进端到端语音识别一直是长期的研究课题。 最近的研究重点是训练端到端自动语音识别（E2E ASR）模型，在不增加推理时评估外部语言模型的额外成本的情况下，获得外部文本数据的性能优势。 在本工作中，我们提出与一组文本到文本的辅助任务联合训练自动语音识别模型，这些任务与它共享解码器和部分编码器。 当我们分别使用960小时的Librispeech和Opensubtitles数据联合训练自动语音识别模型和掩码语言模型时，在测试其他和测试干净数据集上，相对于仅使用自动语音识别的基线模型，我们观察到了16%和20%的词错误率（WER）降低，且在推理时没有任何额外成本；与一个更强的MUTE-L基线模型相比，分别降低了6%和8%，该基线模型使用与我们模型相同的文本数据来训练解码器。 当我们在Librispeech数据上训练掩码语言模型，或者使用机器翻译作为辅助任务时，我们实现了进一步的改进，而不会显著牺牲任务本身的性能。 显示英文摘要

摘要： Improving end-to-end speech recognition by incorporating external text data has been a longstanding research topic. There has been a recent focus on training E2E ASR models that get the performance benefits of external text data without incurring the extra cost of evaluating an external language model at inference time. In this work, we propose training ASR model jointly with a set of text-to-text auxiliary tasks with which it shares a decoder and parts of the encoder. When we jointly train ASR and masked language model with the 960-hour Librispeech and Opensubtitles data respectively, we observe WER reductions of 16% and 20% on test-other and test-clean respectively over an ASR-only baseline without any extra cost at inference time, and reductions of 6% and 8% compared to a stronger MUTE-L baseline which trains the decoder with the same text data as our model. We achieve further improvements when we train masked language model on Librispeech data or when we use machine translation as the auxiliary task, without significantly sacrificing performance on the task itself.