标题： 基于能量模型的最小耗散学习 显示英文标题

标题： Minimal-Dissipation Learning for Energy-Based Models

摘要： 我们证明了持久链能量基础模型（EBM）的近似最大似然估计（MLE）目标函数的偏差恰好等于过阻尼朗之万动力系统中的热力学过量功。 然后我们回答了这样的模型是否可以在有限时间内以最小的过量功，即能量耗散进行训练。 我们发现，具有常数方差的高斯能量函数可以通过仅控制学习率来以最小的过量功进行训练。 这证明了在有限时间内以最小耗散训练持久链EBM是可能的，并且还提供了计算所需的能量下限。 我们将这种最小化过量功的学习过程称为最小耗散学习。 然后我们给出了最优学习率调度到一般势能的推广，并发现它在MLE目标函数上诱导了一个自然梯度流，这是一种众所周知的二阶优化方法。 显示英文摘要

摘要： We show that the bias of the approximate maximum-likelihood estimation (MLE) objective of a persistent chain energy-based model (EBM) is precisely equal to the thermodynamic excess work of an overdamped Langevin dynamical system. We then answer the question of whether such a model can be trained with minimal excess work, that is, energy dissipation, in a finite amount of time. We find that a Gaussian energy function with constant variance can be trained with minimal excess work by controlling only the learning rate. This proves that it is possible to train a persistent chain EBM in a finite amount of time with minimal dissipation and also provides a lower bound on the energy required for the computation. We refer to such a learning process that minimizes the excess work as minimal-dissipation learning. We then provide a generalization of the optimal learning rate schedule to general potentials and find that it induces a natural gradient flow on the MLE objective, a well-known second-order optimization method.