标题： 基准测试与弥合情感冲突以实现多模态情感推理 显示英文标题

标题： Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning

摘要： 尽管在多模态情感推理方面表现出色，现有的多模态大语言模型（MLLMs）通常会忽略涉及情感冲突的场景，其中不同模态的情感线索不一致。为了填补这一空白，我们首先引入CA-MER，一个新的基准，旨在在现实的情感冲突下检验MLLMs。它包含三个子集：视频对齐、音频对齐和一致，其中只有一个或所有模态反映真实情感。然而，在我们的CA-MER上的评估表明，当前最先进的情感MLLMs在情感冲突中系统性地过度依赖音频信号，忽视了视觉模态的关键线索。为了缓解这种偏差，我们提出了MoSEAR，一种参数高效的框架，促进平衡的模态整合。MoSEAR包含两个模块：(1) MoSE，具有正则化门控机制的模态特定专家，减少了微调头部中的模态偏差；以及(2) AR，一种注意力再分配机制，在推理期间重新平衡冻结主干中的模态贡献。我们的框架提供了两个关键优势：它减轻了情感冲突，并提高了在一致样本上的性能——而不会在音频和视觉模态之间产生权衡。在多个基准测试上的实验——包括MER2023、EMER、DFEW和我们的CA-MER——证明MoSEAR实现了最先进性能，特别是在模态冲突条件下。 显示英文摘要

摘要： Despite their strong performance in multimodal emotion reasoning, existing Multimodal Large Language Models (MLLMs) often overlook the scenarios involving emotion conflicts, where emotional cues from different modalities are inconsistent. To fill this gap, we first introduce CA-MER, a new benchmark designed to examine MLLMs under realistic emotion conflicts. It consists of three subsets: video-aligned, audio-aligned, and consistent, where only one or all modalities reflect the true emotion. However, evaluations on our CA-MER reveal that current state-of-the-art emotion MLLMs systematically over-rely on audio signal during emotion conflicts, neglecting critical cues from visual modality. To mitigate this bias, we propose MoSEAR, a parameter-efficient framework that promotes balanced modality integration. MoSEAR consists of two modules: (1)MoSE, modality-specific experts with a regularized gating mechanism that reduces modality bias in the fine-tuning heads; and (2)AR, an attention reallocation mechanism that rebalances modality contributions in frozen backbones during inference. Our framework offers two key advantages: it mitigates emotion conflicts and improves performance on consistent samples-without incurring a trade-off between audio and visual modalities. Experiments on multiple benchmarks-including MER2023, EMER, DFEW, and our CA-MER-demonstrate that MoSEAR achieves state-of-the-art performance, particularly under modality conflict conditions.