标题： 最优集合和ReLU网络的解路径 显示英文标题

标题： Optimal Sets and Solution Paths of ReLU Networks

摘要： 我们开发了一个分析框架，通过将非凸训练问题重新表述为凸程序，来表征最优ReLU神经网络的集合。 我们证明了凸参数化的全局最优解由一个多面体集给出，然后将这种表征扩展到非凸训练目标的最优集。 由于ReLU训练问题的所有驻点都可以表示为子采样凸程序的最优解，我们的工作为非凸目标的所有临界点提供了一个通用表达式。 随后，我们利用这些结果提供了一个用于计算最小网络的最优剪枝算法，建立了ReLU网络正则化路径连续的条件，并开发了最小ReLU网络的敏感性结果。 显示英文摘要

摘要： We develop an analytical framework to characterize the set of optimal ReLU neural networks by reformulating the non-convex training problem as a convex program. We show that the global optima of the convex parameterization are given by a polyhedral set and then extend this characterization to the optimal set of the non-convex training objective. Since all stationary points of the ReLU training problem can be represented as optima of sub-sampled convex programs, our work provides a general expression for all critical points of the non-convex objective. We then leverage our results to provide an optimal pruning algorithm for computing minimal networks, establish conditions for the regularization path of ReLU networks to be continuous, and develop sensitivity results for minimal ReLU networks.