标题： 大型语言模型的指纹 显示英文标题

标题： A Fingerprint for Large Language Models

摘要： 近期进展表明，通过扩展预训练语言模型可以在许多下游任务上实现最先进的性能，促使大型语言模型（LLMs）成为人工智能领域的热门研究课题。然而，由于从头开始训练LLMs的资源密集型特性，保护LLMs的知识产权免受侵权变得紧迫且至关重要。 这激励了本文作者提出了一种针对LLMs的新颖黑盒指纹技术，该技术无需模型训练或微调。 我们首先证明LLMs的输出会形成一个独特的向量空间，与每个模型相关联。 我们将所有权认证问题建模为评估受害者模型空间与嫌疑模型输出空间之间相似性的任务。 为了解决这个问题，我们提出了两种解决方案，其中第一种方案涉及验证嫌疑大型模型的输出是否与受害者模型的输出处于相同的向量空间内，从而实现快速识别模型侵权行为；第二种方案则重构了LLMs输出和受害者模型的向量空间的联合，以应对受害者模型可能遭受的参数高效微调（PEFT）攻击。 实验结果表明，所提出的这项技术在所有权验证和对抗PEFT攻击方面表现出色。 这项工作揭示了LLMs的固有特性，并为黑盒场景下LLMs的所有权验证提供了一个有前景的解决方案，确保了效率、通用性和实用性。 显示英文摘要

摘要： Recent advances show that scaling a pre-trained language model could achieve state-of-the-art performance on many downstream tasks, prompting large language models (LLMs) to become a hot research topic in the field of artificial intelligence. However, due to the resource-intensive nature of training LLMs from scratch, it is urgent and crucial to protect the intellectual property of LLMs against infringement. This has motivated the authors in this paper to propose a novel black-box fingerprinting technique for LLMs, which requires neither model training nor model fine-tuning. We first demonstrate that the outputs of LLMs span a unique vector space associated with each model. We model the problem of ownership authentication as the task of evaluating the similarity between the victim model's space and the output's space of the suspect model. To deal with this problem, we propose two solutions, where the first solution involves verifying whether the outputs of the suspected large model are in the same space as those of the victim model, enabling rapid identification of model infringement, and the second one reconstructs the union of the vector spaces for LLM outputs and the victim model to address situations where the victim model has undergone the Parameter-Efficient Fine-Tuning (PEFT) attacks. Experimental results indicate that the proposed technique achieves superior performance in ownership verification and robustness against PEFT attacks. This work reveals inherent characteristics of LLMs and provides a promising solution for ownership verification of LLMs in black-box scenarios, ensuring efficiency, generality and practicality.