标题： 利用越狱概率攻击和保护多模态大语言模型 显示英文标题

标题： Utilizing Jailbreak Probability to Attack and Safeguard Multimodal LLMs

摘要： 最近，多模态大语言模型（MLLMs）展示了其在理解多模态内容方面的优越能力。 然而，它们仍然容易受到越狱攻击，这些攻击利用其安全对齐中的弱点来生成有害响应。 以前的研究根据响应是否包含恶意内容，将越狱分类为成功或失败。 然而，鉴于MLLM响应的随机性，这种将输入的越狱能力二元分类的方法是不合适的。 基于这一观点，我们引入了越狱概率来量化输入的越狱潜力，这表示当使用该输入提示MLLM时生成恶意响应的可能性。 我们通过多次查询MLLM来近似这个概率。 在使用越狱概率预测网络（JPPN）建模输入隐藏状态与其对应的越狱概率之间的关系后，我们使用连续的越狱概率进行优化。 具体而言，我们提出了基于越狱概率的攻击（JPA），该攻击在输入上优化对抗性扰动以最大化越狱概率。 为了应对攻击，我们还提出了两种防御方法：基于越狱概率的微调（JPF）和基于越狱概率的防御噪声（JPDN），分别在MLLM参数和输入空间中最小化越狱概率。 大量实验表明，(1) 与之前的方法相比，JPA在白盒和黑盒设置下均取得了改进（最高提升28.38%），且扰动范围较小，迭代次数较少。 (2) JPF和JPDN最多可减少超过60%的越狱行为。 上述两个结果都证明了引入越狱概率的重要性，以在输入的越狱能力之间做出细微的区分。 显示英文摘要

摘要： Recently, Multimodal Large Language Models (MLLMs) have demonstrated their superior ability in understanding multimodal contents. However, they remain vulnerable to jailbreak attacks, which exploit weaknesses in their safety alignment to generate harmful responses. Previous studies categorize jailbreaks as successful or failed based on whether responses contain malicious content. However, given the stochastic nature of MLLM responses, this binary classification of an input's ability to jailbreak MLLMs is inappropriate. Derived from this viewpoint, we introduce jailbreak probability to quantify the jailbreak potential of an input, which represents the likelihood that MLLMs generated a malicious response when prompted with this input. We approximate this probability through multiple queries to MLLMs. After modeling the relationship between input hidden states and their corresponding jailbreak probability using Jailbreak Probability Prediction Network (JPPN), we use continuous jailbreak probability for optimization. Specifically, we propose Jailbreak-Probability-based Attack (JPA) that optimizes adversarial perturbations on inputs to maximize jailbreak probability. To counteract attacks, we also propose two defensive methods: Jailbreak-Probability-based Finetuning (JPF) and Jailbreak-Probability-based Defensive Noise (JPDN), which minimizes jailbreak probability in the MLLM parameters and input space, respectively. Extensive experiments show that (1) JPA yields improvements (up to 28.38\%) under both white and black box settings compared to previous methods with small perturbation bounds and few iterations. (2) JPF and JPDN significantly reduce jailbreaks by at most over 60\%. Both of the above results demonstrate the significance of introducing jailbreak probability to make nuanced distinctions among input jailbreak abilities.