标题： ObfusQAte：一种用于评估模糊事实问答中LLM鲁棒性的框架 显示英文标题

标题： ObfusQAte: A Proposed Framework to Evaluate LLM Robustness on Obfuscated Factual Question Answering

摘要： 大型语言模型（LLMs）的迅速普及已显著促进了能够进行事实性问答（QA）的公平人工智能系统的发展。 然而，尚无已知的研究测试LLMs在面对问题的混淆版本时的鲁棒性。 为了系统地评估这些限制，我们提出了一种新技术ObfusQAte，并利用该技术引入了ObfusQA，这是一个全面的、首创的框架，具有多层级的混淆水平，旨在检查LLMs在三个不同维度上的能力：（i）命名实体间接，（ii）干扰项间接，（iii）上下文过载。 通过捕捉语言中的这些细粒度差异，ObfusQA提供了一个全面的基准，用于评估LLM的鲁棒性和适应性。 我们的研究观察到，当LLMs面对这些日益复杂的变体时，倾向于失败或生成幻觉响应。 为了促进这一方向的研究，我们公开了ObfusQAte。 显示英文摘要

摘要： The rapid proliferation of Large Language Models (LLMs) has significantly contributed to the development of equitable AI systems capable of factual question-answering (QA). However, no known study tests the LLMs' robustness when presented with obfuscated versions of questions. To systematically evaluate these limitations, we propose a novel technique, ObfusQAte and, leveraging the same, introduce ObfusQA, a comprehensive, first of its kind, framework with multi-tiered obfuscation levels designed to examine LLM capabilities across three distinct dimensions: (i) Named-Entity Indirection, (ii) Distractor Indirection, and (iii) Contextual Overload. By capturing these fine-grained distinctions in language, ObfusQA provides a comprehensive benchmark for evaluating LLM robustness and adaptability. Our study observes that LLMs exhibit a tendency to fail or generate hallucinated responses when confronted with these increasingly nuanced variations. To foster research in this direction, we make ObfusQAte publicly available.