标题： AstroMMBench：评估天文学中多模态大语言模型能力的基准 显示英文标题

标题： AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy

摘要： 天文学图像解释对于将多模态大语言模型（MLLMs）应用于专业科学任务提出了重大挑战。 现有的基准测试侧重于一般的多模态能力，但未能捕捉天文学数据的复杂性。 为了弥合这一差距，我们引入了AstroMMBench，这是首个全面的基准测试，旨在评估MLLMs在天文学图像理解方面的能力。 AstroMMench包含六个天体物理子领域中的621道选择题，由15位领域专家进行筛选和审查，以确保其质量和相关性。 我们使用AstroMMBench对25种多样化的MLLMs进行了广泛评估，其中包括22种开源模型和3种闭源模型。 结果表明，Ovis2-34B取得了最高的总体准确率（70.5%），即使与强大的闭源模型相比也表现出领先的能力。 在六个天体物理子领域中，性能表现出差异，尤其是在宇宙学和高能天体物理学等领域尤为具有挑战性，而在其他领域如仪器和太阳天体物理学中模型表现相对较好。 这些发现强调了像AstroMMBench这样的领域特定基准在关键评估MLLM性能以及指导其针对科学应用的定向发展中的重要作用。 AstroMMBench提供了一个基础资源和一个动态工具，以推动人工智能和天文学交叉领域的进步。 显示英文摘要

摘要： Astronomical image interpretation presents a significant challenge for applying multimodal large language models (MLLMs) to specialized scientific tasks. Existing benchmarks focus on general multimodal capabilities but fail to capture the complexity of astronomical data. To bridge this gap, we introduce AstroMMBench, the first comprehensive benchmark designed to evaluate MLLMs in astronomical image understanding. AstroMMBench comprises 621 multiple-choice questions across six astrophysical subfields, curated and reviewed by 15 domain experts for quality and relevance. We conducted an extensive evaluation of 25 diverse MLLMs, including 22 open-source and 3 closed-source models, using AstroMMBench. The results show that Ovis2-34B achieved the highest overall accuracy (70.5%), demonstrating leading capabilities even compared to strong closed-source models. Performance showed variations across the six astrophysical subfields, proving particularly challenging in domains like cosmology and high-energy astrophysics, while models performed relatively better in others, such as instrumentation and solar astrophysics. These findings underscore the vital role of domain-specific benchmarks like AstroMMBench in critically evaluating MLLM performance and guiding their targeted development for scientific applications. AstroMMBench provides a foundational resource and a dynamic tool to catalyze advancements at the intersection of AI and astronomy.