标题： 温度：面向大模型推理的热感知三维堆叠架构设计与带宽共享 显示英文标题

标题： Tasa: Thermal-aware 3D-Stacked Architecture Design with Bandwidth Sharing for LLM Inference

摘要： 由于内存密集型操作和有限的硬件带宽，大语言模型中的自回归解码是主要的推理瓶颈。3D堆叠架构是一种有前景的解决方案，其显著提高了内存带宽，通过在逻辑芯片之上垂直堆叠多个DRAM芯片。然而，我们的实验也显示，与2D架构相比，3D堆叠架构在热温度、梯度和可扩展性方面面临更严重的热问题。为了更好地利用3D堆叠架构的潜力，我们提出了Tasa，这是一种具有跨堆栈热优化的异构架构，以平衡温度分布并在热约束下最大化性能。高性能核心专为计算密集型操作设计，而高效率核心用于内存密集型操作，例如注意力层。此外，我们提出了一种带宽共享调度方法，以提高这种异构架构中的带宽利用率。大量的热实验表明，与同构的3D堆叠架构相比，我们的Tasa架构显示出更大的可扩展性，即对于48、60和72核配置，峰值温度分别降低了5.55$\tccentigrade$，9.37$\tccentigrade$和7.91$\tccentigrade$。我们在Llama-65B和GPT-3 66B推理上的实验也表明，相对于GPU基线和最先进的基于PIM的异构LLM加速器，分别获得了2.85倍和2.21倍的速度提升。 显示英文摘要

摘要： The autoregressive decoding in LLMs is the major inference bottleneck due to the memory-intensive operations and limited hardware bandwidth. 3D-stacked architecture is a promising solution with significantly improved memory bandwidth, which vertically stacked multi DRAM dies on top of logic die. However, our experiments also show the 3D-stacked architecture faces severer thermal issues compared to 2D architecture, in terms of thermal temperature, gradient and scalability. To better exploit the potential of 3D-stacked architecture, we present Tasa, a heterogeneous architecture with cross-stack thermal optimizations to balance the temperature distribution and maximize the performance under the thermal constraints. High-performance core is designed for compute-intensive operations, while high-efficiency core is used for memory-intensive operators, e.g. attention layers. Furthermore, we propose a bandwidth sharing scheduling to improve the bandwidth utilization in such heterogeneous architecture. Extensive thermal experiments show that our Tasa architecture demonstrates greater scalability compared with the homogeneous 3D-stacked architecture, i.e. up to 5.55 $\tccentigrade$, 9.37 $\tccentigrade$, and 7.91 $\tccentigrade$ peak temperature reduction for 48, 60, and 72 core configurations. Our experimental for Llama-65B and GPT-3 66B inferences also demonstrate 2.85x and 2.21x speedup are obtained over the GPU baselines and state-of-the-art heterogeneous PIM-based LLM accelerator