标题： 一种无延迟界限的分布式异步广义动量算法 显示英文标题

标题： A Distributed Asynchronous Generalized Momentum Algorithm Without Delay Bounds

摘要： 异步优化算法通常需要延迟界限来证明其收敛性，尽管这些界限在实践中可能难以获得。 不需延迟界限的现有算法通常收敛较慢。 因此，我们引入了一种新颖的分布式广义动量算法，该算法提供快速收敛并允许任意延迟。 它涵盖了Nesterov加速梯度算法和heavy ball算法等。 我们首先开发了该算法参数的条件，以确保渐近收敛。 然后我们展示了其收敛速率与处理器执行的计算和通信次数的函数成线性关系（以我们精确的方式）。 模拟将该算法与梯度下降、heavy ball和Nesterov加速梯度算法进行了比较，在Fashion-MNIST数据集上的分类问题中进行了比较。 在各种具有无界延迟的场景中，广义动量算法的收敛所需的迭代次数至少比梯度下降少71%，比heavy ball算法少41%，比Nesterov加速梯度算法少19%。 显示英文摘要

摘要： Asynchronous optimization algorithms often require delay bounds to prove their convergence, though these bounds can be difficult to obtain in practice. Existing algorithms that do not require delay bounds often converge slowly. Therefore, we introduce a novel distributed generalized momentum algorithm that provides fast convergence and allows arbitrary delays. It subsumes Nesterov's accelerated gradient algorithm and the heavy ball algorithm, among others. We first develop conditions on the parameters of this algorithm that ensure asymptotic convergence. Then we show its convergence rate is linear in a function of the number of computations and communications that processors perform (in a way that we make precise). Simulations compare this algorithm to gradient descent, heavy ball, and Nesterov's accelerated gradient algorithm with a classification problem on the Fashion-MNIST dataset. Across a range of scenarios with unbounded delays, convergence of the generalized momentum algorithm requires at least 71% fewer iterations than gradient descent, 41% fewer iterations than the heavy ball algorithm, and 19% fewer iterations that Nesterov's accelerated gradient algorithm.