标题： SSD卸载对于LLM专家混合权重在能效方面有害 显示英文标题

标题： SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency

摘要： 大型语言模型（LLMs）应用专家混合（MoE）扩展到数万亿参数，但需要大量内存，这促使研究将专家权重从高速但小容量的DRAM（HBM）卸载到密度更高的Flash SSDs。 虽然SSDs提供具有成本效益的容量，但其每比特读取能耗显著高于DRAM。 本文定量分析了在LLLM推理的关键解码阶段将MoE专家权重卸载到SSDs的能耗影响。 我们的分析比较了DeepSeek-R1等模型的SSD、CPU内存（DDR）和HBM存储场景，结果表明，将MoE权重卸载到当前的SSDs会大幅增加每个标记生成的能耗（例如，相比HBM基线最高增加约12倍），主导了总的推理能耗预算。 尽管预取等技术可以有效隐藏访问延迟，但它们无法缓解这一基本能耗惩罚。 我们进一步探讨了未来技术的发展，发现如果Flash读取能耗显著改善，大约一个数量级，MoE模型的固有稀疏性可能使SSDs在能耗上变得可行。 显示英文摘要

摘要： Large Language Models (LLMs) applying Mixture-of-Experts (MoE) scale to trillions of parameters but require vast memory, motivating a line of research to offload expert weights from fast-but-small DRAM (HBM) to denser Flash SSDs. While SSDs provide cost-effective capacity, their read energy per bit is substantially higher than that of DRAM. This paper quantitatively analyzes the energy implications of offloading MoE expert weights to SSDs during the critical decode stage of LLM inference. Our analysis, comparing SSD, CPU memory (DDR), and HBM storage scenarios for models like DeepSeek-R1, reveals that offloading MoE weights to current SSDs drastically increases per-token-generation energy consumption (e.g., by up to ~12x compared to the HBM baseline), dominating the total inference energy budget. Although techniques like prefetching effectively hide access latency, they cannot mitigate this fundamental energy penalty. We further explore future technological scaling, finding that the inherent sparsity of MoE models could potentially make SSDs energy-viable if Flash read energy improves significantly, roughly by an order of magnitude.