标题： FlexRound：基于逐元素除法的可学习量化后训练量化方法 显示英文标题

标题： FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization

摘要： 后训练量化（PTQ）由于不需要完整的训练数据集或端到端训练，因此在资源受限设备上部署深度神经网络方面越来越受欢迎。 由于基于重建每一层或块输出的PTQ方案被证明能有效提高量化模型性能，最近的工作开发了算法来设计和学习一种新的权重舍入方案，以更好地重建每一层或块输出。 在本工作中，我们提出了一种简单但有效的新的权重舍入机制，称为\emph{灵活舍入}，它基于逐元素除法而不是典型的逐元素加法，从而使FlexRound能够联合学习一个通用的量化网格大小以及每个预训练权重的不同尺度。 由于逐元素除法引起的导数倒数规则，FlexRound本质上能够在更新其相应尺度时利用预训练权重，从而根据它们的大小灵活地量化预训练权重。 我们在各种模型和任务上实证验证了FlexRound的有效性。 据我们所知，我们的工作是首次对图像分类、自然语言理解和自然语言生成任务进行全面实验。 此外，我们首次展示了大型语言模型可以高效量化，与半精度基线相比仅对性能产生可忽略的影响，这是通过逐块重建输出实现的。 我们的代码可在\url{https://github.com/onliwad101/FlexRound_LRQ}获取。 显示英文摘要

摘要： Post-training quantization (PTQ) has been gaining popularity for the deployment of deep neural networks on resource-limited devices since unlike quantization-aware training, neither a full training dataset nor end-to-end training is required at all. As PTQ schemes based on reconstructing each layer or block output turn out to be effective to enhance quantized model performance, recent works have developed algorithms to devise and learn a new weight-rounding scheme so as to better reconstruct each layer or block output. In this work, we propose a simple yet effective new weight-rounding mechanism for PTQ, coined \emph{FlexRound}, based on element-wise division instead of typical element-wise addition such that FlexRound enables jointly learning a common quantization grid size as well as a different scale for each pre-trained weight. Thanks to the reciprocal rule of derivatives induced by element-wise division, FlexRound is inherently able to exploit pre-trained weights when updating their corresponding scales, and thus, flexibly quantize pre-trained weights depending on their magnitudes. We empirically validate the efficacy of FlexRound on a wide range of models and tasks. To the best of our knowledge, our work is the first to carry out comprehensive experiments on not only image classification and natural language understanding but also natural language generation. Moreover, we demonstrate, for the first time, that large language models can be efficiently quantized, with only a negligible impact on performance compared to half-precision baselines, achieved by reconstructing the output in a block-by-block manner. Our code is available at \url{https://github.com/onliwad101/FlexRound_LRQ}.