标题： 基于递归学习和策略梯度的稳定认证的在线策略数据驱动LQR 显示英文标题

标题： Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient

摘要： 在本文中，我们研究了一个数据驱动的框架，在动态未知的情况下解决线性二次调节器（LQR）问题，并且还面临为整个学习和控制方案提供稳定性证明的额外挑战。 具体而言，在所提出的在线策略学习框架中，控制输入被应用于实际（未知）的线性系统，并且不断优化。 我们提出了一种称为Relearn LQR的学习和控制过程，该过程结合了递归最小二乘法与基于梯度方法的直接策略搜索。 所得到的方案通过将其建模为反馈互连的非线性动态系统进行分析。 一种基于李雅普诺夫的方法，利用非线性系统的平均化和时标分离理论，使我们能够为整个互联系统提供正式的稳定性保证。 所提出策略的有效性通过数值仿真得到验证，其中Relearn LQR被应用于飞机控制问题，包括静态和漂移参数。 显示英文摘要

摘要： In this paper, we investigate a data-driven framework to solve Linear Quadratic Regulator (LQR) problems when the dynamics is unknown, with the additional challenge of providing stability certificates for the overall learning and control scheme. Specifically, in the proposed on-policy learning framework, the control input is applied to the actual (unknown) linear system while iteratively optimized. We propose a learning and control procedure, termed Relearn LQR, that combines a recursive least squares method with a direct policy search based on the gradient method. The resulting scheme is analyzed by modeling it as a feedback-interconnected nonlinear dynamical system. A Lyapunov-based approach, exploiting averaging and timescale separation theories for nonlinear systems, allows us to provide formal stability guarantees for the whole interconnected scheme. The effectiveness of the proposed strategy is corroborated by numerical simulations, where Relearn LQR is deployed on an aircraft control problem, with both static and drifting parameters.