标题： 面向非均匀和时变通信的FedAvg偏差校正 显示英文标题

标题： Towards Bias Correction of FedAvg over Nonuniform and Time-Varying Communications

摘要： 联邦学习（FL）是一种去中心化的学习框架，其中参数服务器（PS）和一组客户端通过最小化全局目标协同训练模型。 通信带宽是一种稀缺资源；在每一轮中，PS仅聚合一部分客户端的更新。 在本文中，我们关注非凸最小化问题，该问题容易受到PS与客户端之间非均匀且随时间变化的通信故障的影响。 具体来说，在每一轮$t$中，PS与客户端$i$之间的链接以概率$p_i^t$活动，该概率对PS和客户端都是$\textit{unknown}$。 当客户端之间的信道条件异质且随时间变化时，就会出现这种情况。 我们表明，当$p_i^t$不是均匀分布时，广泛采用的FL算法$\textit{Federated Average}$（FedAvg）无法最小化全局目标。 观察到这一点后，我们提出了$\textit{Federated Postponed Broadcast}$（FedPBC），这是FedAvg的一个简单变体。 它与FedAvg的不同之处在于，PS会将全局模型的广播推迟到每一轮结束时。 我们证明FedPBC可以收敛到原始目标的平稳点。 引入的延迟是轻微的，没有明显的减速。 提供了理论分析和数值结果。 在技术方面，推迟全局模型的广播使得在第$t$轮中具有活动链接的客户端之间能够进行隐式的闲聊。 尽管$p_i^t$是时变的，但我们能够通过控制闲聊类型的信息混合误差的技术来限制全局模型动态的扰动。 显示英文摘要

摘要： Federated learning (FL) is a decentralized learning framework wherein a parameter server (PS) and a collection of clients collaboratively train a model via minimizing a global objective. Communication bandwidth is a scarce resource; in each round, the PS aggregates the updates from a subset of clients only. In this paper, we focus on non-convex minimization that is vulnerable to non-uniform and time-varying communication failures between the PS and the clients. Specifically, in each round $t$, the link between the PS and client $i$ is active with probability $p_i^t$, which is $\textit{unknown}$ to both the PS and the clients. This arises when the channel conditions are heterogeneous across clients and are changing over time. We show that when the $p_i^t$'s are not uniform, $\textit{Federated Average}$ (FedAvg) -- the most widely adopted FL algorithm -- fails to minimize the global objective. Observing this, we propose $\textit{Federated Postponed Broadcast}$ (FedPBC) which is a simple variant of FedAvg. It differs from FedAvg in that the PS postpones broadcasting the global model till the end of each round. We show that FedPBC converges to a stationary point of the original objective. The introduced staleness is mild and there is no noticeable slowdown. Both theoretical analysis and numerical results are provided. On the technical front, postponing the global model broadcasts enables implicit gossiping among the clients with active links at round $t$. Despite $p_i^t$'s are time-varying, we are able to bound the perturbation of the global model dynamics via the techniques of controlling the gossip-type information mixing errors.