标题： MedPatch：基于置信度的多阶段融合用于多模态临床数据 显示英文标题

标题： MedPatch: Confidence-Guided Multi-Stage Fusion for Multimodal Clinical Data

摘要： 临床决策依赖于跨多种数据模态的信息整合，例如临床时间序列、医学图像和文本报告。 与其他领域相比，现实世界的医疗数据在本质上是异构的，规模有限，并且由于缺少模态而稀疏。 这显著限制了模型在临床预测任务中的性能。 受临床工作流程的启发，我们引入了MedPatch，这是一种多阶段多模态融合架构，通过置信度引导的补丁方法无缝整合多种模态。 MedPatch包含三个主要组件：(i) 一种多阶段融合策略，同时利用联合融合和晚期融合，(ii) 一种缺失感知模块，用于处理具有缺失模态的稀疏样本，(iii) 一种联合融合模块，根据校准的单模态令牌级置信度对潜在令牌补丁进行聚类。 我们使用包含临床时间序列数据、胸部X光图像、放射科报告和出院记录的真实世界数据，在两个基准任务上评估了MedPatch，即住院死亡率预测和临床状况分类，这些数据来自MIMIC-IV、MIMIC-CXR和MIMIC-Notes数据集。 与现有基线相比，MedPatch取得了最先进的性能。 我们的工作突显了置信度引导的多阶段融合在解决多模态数据异构性方面的有效性，并为临床预测任务建立了新的最先进基准结果。 显示英文摘要

摘要： Clinical decision-making relies on the integration of information across various data modalities, such as clinical time-series, medical images and textual reports. Compared to other domains, real-world medical data is heterogeneous in nature, limited in size, and sparse due to missing modalities. This significantly limits model performance in clinical prediction tasks. Inspired by clinical workflows, we introduce MedPatch, a multi-stage multimodal fusion architecture, which seamlessly integrates multiple modalities via confidence-guided patching. MedPatch comprises three main components: (i) a multi-stage fusion strategy that leverages joint and late fusion simultaneously, (ii) a missingness-aware module that handles sparse samples with missing modalities, (iii) a joint fusion module that clusters latent token patches based on calibrated unimodal token-level confidence. We evaluated MedPatch using real-world data consisting of clinical time-series data, chest X-ray images, radiology reports, and discharge notes extracted from the MIMIC-IV, MIMIC-CXR, and MIMIC-Notes datasets on two benchmark tasks, namely in-hospital mortality prediction and clinical condition classification. Compared to existing baselines, MedPatch achieves state-of-the-art performance. Our work highlights the effectiveness of confidence-guided multi-stage fusion in addressing the heterogeneity of multimodal data, and establishes new state-of-the-art benchmark results for clinical prediction tasks.