标题： 逆向工程NLI：自然语言推理的元推理属性研究 显示英文标题

标题： Reverse-engineering NLI: A study of the meta-inferential properties of Natural Language Inference

摘要： 自然语言推理（NLI）一直是评估自然语言理解语言模型的重要任务，但该任务的逻辑特性了解不足且常被错误描述。 理解NLI所捕捉的推理概念对于解释模型在该任务上的表现至关重要。 在本文中，我们提出了NLI标签集的三种可能解读，并对它们所蕴含的元推理特性进行了全面分析。 专注于SNLI数据集，我们利用（1）具有共同前提的NLI条目和（2）由大语言模型生成的条目，来评估在SNLI上训练的模型的元推理一致性，并得出关于数据集中编码的逻辑关系解读的见解。 显示英文摘要

摘要： Natural Language Inference (NLI) has been an important task for evaluating language models for Natural Language Understanding, but the logical properties of the task are poorly understood and often mischaracterized. Understanding the notion of inference captured by NLI is key to interpreting model performance on the task. In this paper we formulate three possible readings of the NLI label set and perform a comprehensive analysis of the meta-inferential properties they entail. Focusing on the SNLI dataset, we exploit (1) NLI items with shared premises and (2) items generated by LLMs to evaluate models trained on SNLI for meta-inferential consistency and derive insights into which reading of the logical relations is encoded by the dataset.