标题： 无导数顺序二次规划用于等式约束随机优化 显示英文标题

标题： Derivative-Free Sequential Quadratic Programming for Equality-Constrained Stochastic Optimization

摘要： 我们考虑解决具有随机目标函数和确定性等式约束的非线性优化问题，假设对于目标函数和约束条件仅能获得零阶信息，并且目标函数还受到随机采样噪声的影响。 在这一设定下，我们提出了一种无导数随机顺序二次规划（DF-SSQP）方法。 由于缺乏导数信息，我们采用同时扰动随机逼近（SPSA）技术来随机估计目标函数和约束条件的梯度和海森矩阵。 这种方法在每次迭代步骤中只需要一个与维度无关的零阶评估数量——最少为八个。 我们无导数方法与现有基于导数的SSQP方法之间的一个关键区别在于，由随机零阶近似引入的复杂随机偏差，该偏差出现在目标函数和约束条件的梯度和海森矩阵估计中。 为了解决这个问题，我们引入了一种基于动量风格估计器的在线去偏技术，该技术适当聚合过去的梯度和海森矩阵估计以减少随机噪声，同时通过移动平均方案避免过高的内存成本。 在标准假设下，我们建立了所提出的DF-SSQP方法的全局几乎必然收敛性。 值得注意的是，我们进一步通过证明重新缩放的迭代点表现出渐近正态性，其极限协方差矩阵类似于基于导数的方法实现的最小最大最优协方差矩阵，尽管由于缺乏导数信息而更大，从而补充了全局分析。 我们的局部分析使得可以利用DF-SSQP进行模型参数的在线统计推断。 在基准非线性问题上的数值实验展示了DF-SSQP的全局和局部行为。 显示英文摘要

摘要： We consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints, assuming that only zero-order information is available for both the objective and constraints, and that the objective is also subject to random sampling noise. Under this setting, we propose a Derivative-Free Stochastic Sequential Quadratic Programming (DF-SSQP) method. Due to the lack of derivative information, we adopt a simultaneous perturbation stochastic approximation (SPSA) technique to randomly estimate the gradients and Hessians of both the objective and constraints. This approach requires only a dimension-independent number of zero-order evaluations -- as few as eight -- at each iteration step. A key distinction between our derivative-free and existing derivative-based SSQP methods lies in the intricate random bias introduced into the gradient and Hessian estimates of the objective and constraints, brought by stochastic zero-order approximations. To address this issue, we introduce an online debiasing technique based on momentum-style estimators that properly aggregate past gradient and Hessian estimates to reduce stochastic noise, while avoiding excessive memory costs via a moving averaging scheme. Under standard assumptions, we establish the global almost-sure convergence of the proposed DF-SSQP method. Notably, we further complement the global analysis with local convergence guarantees by demonstrating that the rescaled iterates exhibit asymptotic normality, with a limiting covariance matrix resembling the minimax optimal covariance achieved by derivative-based methods, albeit larger due to the absence of derivative information. Our local analysis enables online statistical inference of model parameters leveraging DF-SSQP. Numerical experiments on benchmark nonlinear problems demonstrate both the global and local behavior of DF-SSQP.