标题： 破解代码：通过系统越狱攻击对AI代码代理的安全性评估 显示英文标题

标题： Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks

摘要： 代码能力强大的大型语言模型（LLM）代理正越来越多地嵌入软件工程工作流中，它们可以读取、编写和执行代码，使安全绕过（“越狱”）攻击的威胁超出了仅文本的环境。 以往的评估强调拒绝或有害文本检测，而未明确代理是否实际编译并运行恶意程序。 我们提出了JAWS-BENCH（跨工作区越狱），这是一个涵盖三种逐步升级的工作区模式的基准，反映了攻击者的能力建设：空（JAWS-0）、单文件（JAWS-1）和多文件（JAWS-M）。 我们还配套了一个分层的、可执行的Judge框架，用于测试（i）合规性，（ii）攻击成功率，（iii）语法正确性，以及（iv）运行时可执行性，超越拒绝行为来衡量可部署的危害。 使用来自五个家族的七种LLMs作为后端，我们发现，在JAWS-0的仅提示条件下，代码代理平均接受61%的攻击；其中58%是有害的，52%可以解析，27%能够端到端运行。 转向JAWS-1的单文件模式，对于能力强的模型，合规性达到约100%，平均攻击成功率（ASR）约为71%；多文件模式（JAWS-M）将平均ASR提高到约75%，其中32%的攻击代码可以立即部署。 在不同模型中，将LLM封装成代理会显著增加脆弱性——ASR提高了1.6倍——因为初始拒绝通常在后续的规划/工具使用步骤中被推翻。 类别级别的分析确定了哪些攻击类别最易受攻击且最容易部署，而其他类别则表现出较大的执行差距。 这些发现促使我们提出执行感知的防御措施、代码上下文的安全过滤器，以及在代理的多步骤推理和工具使用过程中保持拒绝决策的机制。 显示英文摘要

摘要： Code-capable large language model (LLM) agents are increasingly embedded into software engineering workflows where they can read, write, and execute code, raising the stakes of safety-bypass ("jailbreak") attacks beyond text-only settings. Prior evaluations emphasize refusal or harmful-text detection, leaving open whether agents actually compile and run malicious programs. We present JAWS-BENCH (Jailbreaks Across WorkSpaces), a benchmark spanning three escalating workspace regimes that mirror attacker capability: empty (JAWS-0), single-file (JAWS-1), and multi-file (JAWS-M). We pair this with a hierarchical, executable-aware Judge Framework that tests (i) compliance, (ii) attack success, (iii) syntactic correctness, and (iv) runtime executability, moving beyond refusal to measure deployable harm. Using seven LLMs from five families as backends, we find that under prompt-only conditions in JAWS-0, code agents accept 61% of attacks on average; 58% are harmful, 52% parse, and 27% run end-to-end. Moving to single-file regime in JAWS-1 drives compliance to ~ 100% for capable models and yields a mean ASR (Attack Success Rate) ~ 71%; the multi-file regime (JAWS-M) raises mean ASR to ~ 75%, with 32% instantly deployable attack code. Across models, wrapping an LLM in an agent substantially increases vulnerability -- ASR raises by 1.6x -- because initial refusals are frequently overturned during later planning/tool-use steps. Category-level analyses identify which attack classes are most vulnerable and most readily deployable, while others exhibit large execution gaps. These findings motivate execution-aware defenses, code-contextual safety filters, and mechanisms that preserve refusal decisions throughout the agent's multi-step reasoning and tool use.