标题： ReGate：在神经处理单元中实现电源门控 显示英文标题

标题： ReGate: Enabling Power Gating in Neural Processing Units

摘要： 神经处理单元（NPU）的能效在开发可持续数据中心中起着关键作用。 我们的研究显示，不同代的NPU芯片中，30%-72%的能耗是由静态功耗引起的，这是由于现代NPU芯片缺乏电源管理支持。 在本文中，我们提出了ReGate，它通过软硬件协同设计，实现了对NPU芯片中每个硬件组件的细粒度电源门控。 与传统用于通用处理器的电源门控技术不同，由于硬件架构和程序执行模型的根本差异，在NPU中实现电源门控面临独特的挑战。 为了解决这些挑战，我们仔细研究了NPU芯片中每个组件的电源门控机会，并决定最适合的电源管理方案（即硬件管理或软件管理的电源门控）。 具体来说，对于具有确定性执行模式的脉动阵列（SAs），ReGate根据SAs中的固有数据流执行，在处理元素（PE）的粒度上实现周期级的电源门控。 对于具有较长空闲间隔的芯片间互连（ICI）和HBM控制器，ReGate采用了一种轻量级的基于硬件的空闲检测机制。 对于向量单元和SRAM，它们的空闲时间根据工作负载模式变化较大，ReGate扩展了NPU指令集架构（ISA），并允许编译器等软件管理电源门控。 在生产级别的NPU模拟器上实现后，我们展示了ReGate可以将NPU芯片的能耗降低多达32.8%（平均15.5%），对AI工作负载性能的影响可以忽略不计。 电源门控逻辑的硬件实现对NPU芯片的开销小于3.3%。 显示英文摘要

摘要： The energy efficiency of neural processing units (NPU) is playing a critical role in developing sustainable data centers. Our study with different generations of NPU chips reveals that 30%-72% of their energy consumption is contributed by static power dissipation, due to the lack of power management support in modern NPU chips. In this paper, we present ReGate, which enables fine-grained power-gating of each hardware component in NPU chips with hardware/software co-design. Unlike conventional power-gating techniques for generic processors, enabling power-gating in NPUs faces unique challenges due to the fundamental difference in hardware architecture and program execution model. To address these challenges, we carefully investigate the power-gating opportunities in each component of NPU chips and decide the best-fit power management scheme (i.e., hardware- vs. software-managed power gating). Specifically, for systolic arrays (SAs) that have deterministic execution patterns, ReGate enables cycle-level power gating at the granularity of processing elements (PEs) following the inherent dataflow execution in SAs. For inter-chip interconnect (ICI) and HBM controllers that have long idle intervals, ReGate employs a lightweight hardware-based idle-detection mechanism. For vector units and SRAM whose idle periods vary significantly depending on workload patterns, ReGate extends the NPU ISA and allows software like compilers to manage the power gating. With implementation on a production-level NPU simulator, we show that ReGate can reduce the energy consumption of NPU chips by up to 32.8% (15.5% on average), with negligible impact on AI workload performance. The hardware implementation of power-gating logic introduces less than 3.3% overhead in NPU chips.