标题： SecureBERT 2.0：网络安全智能的先进语言模型 显示英文标题

标题： SecureBERT 2.0: Advanced Language Model for Cybersecurity Intelligence

摘要： 有效分析网络安全和威胁情报数据需要能够解释专业术语、复杂文档结构以及自然语言和源代码之间相互依赖关系的语言模型。 仅编码器的变压器架构提供了高效且稳健的表示，支持语义搜索、技术实体提取和语义分析等关键任务，这些任务对于自动化威胁检测、事件分类和漏洞评估至关重要。 然而，通用语言模型通常缺乏实现高精度所需的领域特定适应性。 我们提出了SecureBERT 2.0，这是一个专为网络安全应用而构建的增强型仅编码器语言模型。 利用ModernBERT架构，SecureBERT 2.0引入了改进的长上下文建模和分层编码，使对扩展和异构文档（包括威胁报告和源代码工件）的有效处理成为可能。 在比其前身大十三倍以上的领域特定语料库上预训练，该语料库包含来自各种现实世界来源的超过130亿个文本标记和5300万个代码标记，SecureBERT 2.0在多个网络安全基准测试中达到了最先进的性能。 实验结果表明，在威胁情报的语义搜索、语义分析、网络安全特定的命名实体识别以及网络安全领域内的代码自动漏洞检测方面有显著提升。 显示英文摘要

摘要： Effective analysis of cybersecurity and threat intelligence data demands language models that can interpret specialized terminology, complex document structures, and the interdependence of natural language and source code. Encoder-only transformer architectures provide efficient and robust representations that support critical tasks such as semantic search, technical entity extraction, and semantic analysis, which are key to automated threat detection, incident triage, and vulnerability assessment. However, general-purpose language models often lack the domain-specific adaptation required for high precision. We present SecureBERT 2.0, an enhanced encoder-only language model purpose-built for cybersecurity applications. Leveraging the ModernBERT architecture, SecureBERT 2.0 introduces improved long-context modeling and hierarchical encoding, enabling effective processing of extended and heterogeneous documents, including threat reports and source code artifacts. Pretrained on a domain-specific corpus more than thirteen times larger than its predecessor, comprising over 13 billion text tokens and 53 million code tokens from diverse real-world sources, SecureBERT 2.0 achieves state-of-the-art performance on multiple cybersecurity benchmarks. Experimental results demonstrate substantial improvements in semantic search for threat intelligence, semantic analysis, cybersecurity-specific named entity recognition, and automated vulnerability detection in code within the cybersecurity domain.