标题： AutoClip: 针对源分离网络的自适应梯度裁剪 显示英文标题

标题： AutoClip: Adaptive Gradient Clipping for Source Separation Networks

摘要： 裁剪梯度是一种改进梯度下降的已知方法，但需要手动选择裁剪阈值超参数。 我们提出了 AutoClip，一种基于训练过程中观察到的梯度范数历史，自动且自适应地选择梯度裁剪阈值的简单方法。 实验结果显示，应用 AutoClip 可以提升音频源分离网络的泛化性能。 观察使用和未使用 AutoClip 训练的分离网络的训练动态表明，AutoClip 引导优化进入了损失函数景观更平滑的部分。 AutoClip 实现非常简单，并且可以轻松集成到多个领域的各种应用程序中。 显示英文摘要

摘要： Clipping the gradient is a known approach to improving gradient descent, but requires hand selection of a clipping threshold hyperparameter. We present AutoClip, a simple method for automatically and adaptively choosing a gradient clipping threshold, based on the history of gradient norms observed during training. Experimental results show that applying AutoClip results in improved generalization performance for audio source separation networks. Observation of the training dynamics of a separation network trained with and without AutoClip show that AutoClip guides optimization into smoother parts of the loss landscape. AutoClip is very simple to implement and can be integrated readily into a variety of applications across multiple domains.