标题： BanglaMATH：针对6、7和8年级的Bangla基准数据集，用于测试LLM的数学推理能力 显示英文标题

标题： BanglaMATH : A Bangla benchmark dataset for testing LLM mathematical reasoning at grades 6, 7, and 8

摘要： 大型语言模型（LLMs）在支持数学推理方面具有巨大潜力，其在教育和人工智能研究中的应用日益增多。 然而，大多数现有的基准测试仅限于英语，这对低资源语言造成了显著的差距。 例如，孟加拉语有近2.5亿人使用，这些用户将从具备母语流利能力的LLMs中受益。 为了解决这个问题，我们提出了BanglaMATH，这是一个包含1.7k个孟加拉语数学应用题的数据集，涵盖算术、代数、几何和逻辑推理等主题，数据来源于孟加拉语小学课本，并标注了年级水平和推理步骤数量等信息。 我们设计BanglaMATH来评估商业和开源LLMs在孟加拉语中的数学能力，我们发现Gemini 2.5 Flash和DeepSeek V3是唯一达到强大性能的模型，在三个小学年级中达到了$\ge$80%的准确率。 此外，我们通过在原始问题中添加干扰信息，并将问题翻译成英语，评估了这些表现最佳的LLMs的鲁棒性和语言偏差。 我们展示了这两种LLMs无法保持鲁棒性，并在孟加拉语中表现出显著的性能偏差。 我们的研究表明，LLMs在处理低资源语言中的算术和数学推理方面仍存在当前局限性，并强调了对多语言和公平数学理解进一步研究的必要性。 数据集链接：\href{https://github.com/TabiaTanzin/BanglaMATH-A-Bangla-benchmark-dataset-for-testing-LLM-mathematical-reasoning-at-grades-6-7-and-8.git}{https://github.com/BanglaMATH} 显示英文摘要

摘要： Large Language Models (LLMs) have tremendous potential to play a key role in supporting mathematical reasoning, with growing use in education and AI research. However, most existing benchmarks are limited to English, creating a significant gap for low-resource languages. For example, Bangla is spoken by nearly 250 million people who would collectively benefit from LLMs capable of native fluency. To address this, we present BanglaMATH, a dataset of 1.7k Bangla math word problems across topics such as Arithmetic, Algebra, Geometry, and Logical Reasoning, sourced from Bangla elementary school workbooks and annotated with details like grade level and number of reasoning steps. We have designed BanglaMATH to evaluate the mathematical capabilities of both commercial and open-source LLMs in Bangla, and we find that Gemini 2.5 Flash and DeepSeek V3 are the only models to achieve strong performance, with $\ge$ 80\% accuracy across three elementary school grades. Furthermore, we assess the robustness and language bias of these top-performing LLMs by augmenting the original problems with distracting information, and translating the problems into English. We show that both LLMs fail to maintain robustness and exhibit significant performance bias in Bangla. Our study underlines current limitations of LLMs in handling arithmetic and mathematical reasoning in low-resource languages, and highlights the need for further research on multilingual and equitable mathematical understanding. Dataset link: \href{https://github.com/TabiaTanzin/BanglaMATH-A-Bangla-benchmark-dataset-for-testing-LLM-mathematical-reasoning-at-grades-6-7-and-8.git}{https://github.com/BanglaMATH}