标题： 嵌入信任：语义各向同性预测长文本生成中的非事实性 显示英文标题

标题： Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation

摘要： 为了在需要实质性准确响应的高风险应用领域部署大型语言模型（LLMs），我们需要可靠且计算成本低廉的方法来评估LLMs生成的长文本响应的可信度。 然而，现有的方法通常依赖于逐项事实核查，这在长文本响应中计算成本高昂且脆弱。 在本工作中，我们引入了语义各向同性——在单位球面上归一化文本嵌入的均匀程度——并用它来评估LLMs生成的长文本响应的可信度。 为此，我们生成多个长文本响应，对其进行嵌入，并将这些响应的语义各向同性水平估计为嵌入在单位球面上的角度分散度。 我们发现，更高的语义各向同性——即更大的嵌入分散度——可靠地表明样本之间的事实一致性较低。 我们的方法不需要标记数据、不需要微调，也不需要超参数选择，并且可以与开放权重或封闭权重的嵌入模型一起使用。 在多个领域中，我们的方法仅使用少量样本就能持续优于现有方法，在预测长文本响应中的非事实性方面——为将可信度评估整合到实际的LLM工作流中提供了一种实用且低成本的方法。 显示英文摘要

摘要： To deploy large language models (LLMs) in high-stakes application domains that require substantively accurate responses to open-ended prompts, we need reliable, computationally inexpensive methods that assess the trustworthiness of long-form responses generated by LLMs. However, existing approaches often rely on claim-by-claim fact-checking, which is computationally expensive and brittle in long-form responses to open-ended prompts. In this work, we introduce semantic isotropy -- the degree of uniformity across normalized text embeddings on the unit sphere -- and use it to assess the trustworthiness of long-form responses generated by LLMs. To do so, we generate several long-form responses, embed them, and estimate the level of semantic isotropy of these responses as the angular dispersion of the embeddings on the unit sphere. We find that higher semantic isotropy -- that is, greater embedding dispersion -- reliably signals lower factual consistency across samples. Our approach requires no labeled data, no fine-tuning, and no hyperparameter selection, and can be used with open- or closed-weight embedding models. Across multiple domains, our method consistently outperforms existing approaches in predicting nonfactuality in long-form responses using only a handful of samples -- offering a practical, low-cost approach for integrating trust assessment into real-world LLM workflows.