标题： 基于邻域采样的动量随机方法用于训练图神经网络 显示英文标题

标题： Neighbor-Sampling Based Momentum Stochastic Methods for Training Graph Neural Networks

摘要： 图卷积网络（GCNs）是图表示学习的强大工具。 由于GCNs采用递归的邻域聚合，高效的训练方法缺乏理论保证，或者缺少现代深度学习算法中的重要实用元素，例如自适应性和动量。 在本文中，我们提出几种基于邻居采样（NS）的Adam类型随机方法，用于解决非凸GCN训练问题。 我们利用[1]提出的控制变量技术来减少由邻居采样引起的随机误差。 在Adam类型方法的标准假设下，我们证明了我们的方法具有最优收敛速率。 此外，我们在节点分类任务上进行了大量数值实验，使用了多个基准数据集。 结果表明，我们的方法在经典NS基础上的SGD上表现更优，后者也使用了控制变量技术，尤其是在大规模图数据集上。 我们的代码可在https://github.com/RPI-OPT/CV-ADAM-GNN 获取。 显示英文摘要

摘要： Graph convolutional networks (GCNs) are a powerful tool for graph representation learning. Due to the recursive neighborhood aggregations employed by GCNs, efficient training methods suffer from a lack of theoretical guarantees or are missing important practical elements from modern deep learning algorithms, such as adaptivity and momentum. In this paper, we present several neighbor-sampling (NS) based Adam-type stochastic methods for solving a nonconvex GCN training problem. We utilize the control variate technique proposed by [1] to reduce the stochastic error caused by neighbor sampling. Under standard assumptions for Adam-type methods, we show that our methods enjoy the optimal convergence rate. In addition, we conduct extensive numerical experiments on node classification tasks with several benchmark datasets. The results demonstrate superior performance of our methods over classic NS-based SGD that also uses the control-variate technique, especially for large-scale graph datasets. Our code is available at https://github.com/RPI-OPT/CV-ADAM-GNN .