标题： 打开黑箱：通过语义共振架构实现可解释的大型语言模型 显示英文标题

标题： Opening the Black Box: Interpretable LLMs via Semantic Resonance Architecture

摘要： 大型语言模型（LLMs）表现出色，但仍然难以解释。 混合专家（MoE）模型通过稀疏激活提高效率，但通常依赖于不透明的、学习到的门控函数。 虽然基于相似性的路由（余弦路由器）已被用于训练稳定性，但其在固有可解释性方面的潜力尚未被充分挖掘。 我们引入了语义共振架构（SRA），这是一种旨在确保路由决策本身可解释的MoE方法。 SRA用语义共振室（CSR）模块取代了学习到的门控，该模块根据与可训练语义锚点的余弦相似性进行标记路由。 我们还引入了一种新的分散损失，鼓励锚点之间的正交性以强制多样化的专业化。 在WikiText-103上的实验表明，SRA在匹配的活跃参数约束（29.0M）下实现了验证困惑度13.41，优于密集基线（14.13）和标准MoE基线（13.53）。 至关重要的是，SRA表现出优越的专家利用率（1.0%的死亡专家 vs 标准MoE中的14.8%），并发展出独特且语义一致的专业化模式，这与标准MoEs中观察到的噪声专业化不同。 这项工作确立了语义路由作为一种稳健的方法论，用于构建更透明和可控的语言模型。 显示英文摘要

摘要： Large language models (LLMs) achieve remarkable performance but remain difficult to interpret. Mixture-of-Experts (MoE) models improve efficiency through sparse activation, yet typically rely on opaque, learned gating functions. While similarity-based routing (Cosine Routers) has been explored for training stabilization, its potential for inherent interpretability remains largely untapped. We introduce the Semantic Resonance Architecture (SRA), an MoE approach designed to ensure that routing decisions are inherently interpretable. SRA replaces learned gating with a Chamber of Semantic Resonance (CSR) module, which routes tokens based on cosine similarity with trainable semantic anchors. We also introduce a novel Dispersion Loss that encourages orthogonality among anchors to enforce diverse specialization. Experiments on WikiText-103 demonstrate that SRA achieves a validation perplexity of 13.41, outperforming both a dense baseline (14.13) and a Standard MoE baseline (13.53) under matched active parameter constraints (29.0M). Crucially, SRA exhibits superior expert utilization (1.0% dead experts vs. 14.8% in the Standard MoE) and develops distinct, semantically coherent specialization patterns, unlike the noisy specialization observed in standard MoEs. This work establishes semantic routing as a robust methodology for building more transparent and controllable language models.