标题： Branchformer：并行MLP-Attention架构以捕获语音识别和理解中的局部和全局上下文 显示英文标题

标题： Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding

摘要： Conformer在许多语音处理任务中已被证明是有效的。 它结合了使用卷积提取局部依赖性和使用自注意机制提取全局依赖性的优点。 受此启发，我们提出了一种更灵活、可解释和可定制的编码器替代方案Branchformer，在端到端语音处理中并行分支用于建模各种范围的依赖关系。 在每个编码器层中，一个分支使用自注意机制或其变体来捕捉长距离依赖性，而另一个分支则利用带有卷积门控的MLP模块（cgMLP）来提取局部关系。 我们在多个语音识别和口语语言理解基准上进行了实验。 结果表明，我们的模型优于Transformer和cgMLP。 它还与Conformer取得的最先进结果相当或更好。 此外，我们展示了由于双分支架构而可以减少计算的各种策略，包括在单个训练模型中具有可变推理复杂性的能力。 合并分支所学习的权重表明了不同层中局部和全局依赖性的使用方式，这有助于模型设计。 显示英文摘要

摘要： Conformer has proven to be effective in many speech processing tasks. It combines the benefits of extracting local dependencies using convolutions and global dependencies using self-attention. Inspired by this, we propose a more flexible, interpretable and customizable encoder alternative, Branchformer, with parallel branches for modeling various ranged dependencies in end-to-end speech processing. In each encoder layer, one branch employs self-attention or its variant to capture long-range dependencies, while the other branch utilizes an MLP module with convolutional gating (cgMLP) to extract local relationships. We conduct experiments on several speech recognition and spoken language understanding benchmarks. Results show that our model outperforms both Transformer and cgMLP. It also matches with or outperforms state-of-the-art results achieved by Conformer. Furthermore, we show various strategies to reduce computation thanks to the two-branch architecture, including the ability to have variable inference complexity in a single trained model. The weights learned for merging branches indicate how local and global dependencies are utilized in different layers, which benefits model designing.