标题： 基于编码器-解码器的多模态说话人变化检测 显示英文标题

标题： Encoder-decoder multimodal speaker change detection

摘要： 说话人变化检测（SCD）任务，即检测输入中说话人发生变化的点，在多个应用中至关重要。一些研究仅使用音频输入解决了SCD任务，但性能有限。最近，利用文本模态与音频相结合的多模态SCD（MMSCD）模型显示出更好的性能。本研究提出的模型基于两个主要建议：一种新的模态融合机制和采用编码器-解码器架构。不同于以前从与单个单词对齐的极短音频片段中提取说话人嵌入的MMSCD工作，我们使用从1.5秒音频中提取的说话人嵌入。一个Transformer解码器层进一步提高了仅编码器的MMSCD模型的性能。所提出的模型在报告SCD性能的研究中达到了最先进的结果，并且与最近通过人工转录结合SCD和自动语音识别的工作相当。 显示英文摘要

摘要： The task of speaker change detection (SCD), which detects points where speakers change in an input, is essential for several applications. Several studies solved the SCD task using audio inputs only and have shown limited performance. Recently, multimodal SCD (MMSCD) models, which utilise text modality in addition to audio, have shown improved performance. In this study, the proposed model are built upon two main proposals, a novel mechanism for modality fusion and the adoption of a encoder-decoder architecture. Different to previous MMSCD works that extract speaker embeddings from extremely short audio segments, aligned to a single word, we use a speaker embedding extracted from 1.5s. A transformer decoder layer further improves the performance of an encoder-only MMSCD model. The proposed model achieves state-of-the-art results among studies that report SCD performance and is also on par with recent work that combines SCD with automatic speech recognition via human transcription.