标题： DistilXLSR：一种轻量级跨语言语音表示模型 显示英文标题

标题： DistilXLSR: A Light Weight Cross-Lingual Speech Representation Model

摘要： 多语言自监督语音表征模型极大地提升了低资源语言的语音识别性能，同时这些庞大模型的压缩也成为其工业应用的关键先决条件。 本文提出了一种名为DistilXLSR的蒸馏跨语言语音表征模型。通过随机打乱现有语音的音素，减少语言信息，并仅使用英语数据来蒸馏跨语言模型。我们还设计了一种层跳跃初始化方法，以充分利用教师模型的预训练权重。实验结果显示，在两种教师模型和15种低资源语言上的测试表明，我们的方法可以在保持跨语言表征能力的同时将参数量减少50%。此外，我们的方法被证明对各种语言/教师模型具有通用性，并有可能提升英语预训练模型的跨语言性能。 显示英文摘要

摘要： Multilingual self-supervised speech representation models have greatly enhanced the speech recognition performance for low-resource languages, and the compression of these huge models has also become a crucial prerequisite for their industrial application. In this paper, we propose DistilXLSR, a distilled cross-lingual speech representation model. By randomly shuffling the phonemes of existing speech, we reduce the linguistic information and distill cross-lingual models using only English data. We also design a layer-jumping initialization method to fully leverage the teacher's pre-trained weights. Experiments on 2 kinds of teacher models and 15 low-resource languages show that our method can reduce the parameters by 50% while maintaining cross-lingual representation ability. Our method is proven to be generalizable to various languages/teacher models and has the potential to improve the cross-lingual performance of the English pre-trained models.