标题： SOP：一种适用于视觉-语言-动作模型的可扩展在线微调系统 显示英文标题

标题： SOP: A Scalable Online Post-Training System for Vision-Language-Action Models

摘要： 视觉-语言-动作（VLA）模型通过大规模预训练实现了强大的泛化能力，但实际部署需要除了广泛泛化之外的专家级任务熟练度。 现有的VLA模型后训练方法通常是离线的、单机器人的或任务特定的，限制了有效的策略内适应和从真实世界交互中可扩展的学习。 我们引入了一个可扩展的在线后训练（SOP）系统，该系统能够在物理世界中直接对通用VLA模型进行在线、分布式、多任务后训练。 SOP通过闭环架构将执行和学习紧密结合，在这种架构中，一支机器人队伍持续地将策略内的经验以及人类干预信号传输到一个中心云学习器，并异步接收更新后的策略。 这种设计支持及时的策略内校正，通过并行部署扩大经验收集，并在适应过程中保持泛化性。 SOP对后训练算法的选择是无偏的；我们使用交互式模仿学习（HG-DAgger）和强化学习（RECAP）对其进行实例化。 在一系列真实世界的操作任务中，包括衣物折叠、箱子组装和杂货补货，我们展示了SOP显著提升了大型预训练VLA模型的性能，同时在任务之间保持单一共享策略。 在数小时内的真实世界交互中可以实现有效的后训练，性能随着舰队中机器人数量的增加而近似线性增长。 这些结果表明，将在线学习与舰队规模的部署紧密结合起来，对于在物理世界中实现高效、可靠和可扩展的通用机器人策略后训练至关重要。 显示英文摘要

摘要： Vision-language-action (VLA) models achieve strong generalization through large-scale pre-training, but real-world deployment requires expert-level task proficiency in addition to broad generality. Existing post-training approaches for VLA models are typically offline, single-robot, or task-specific, limiting effective on-policy adaptation and scalable learning from real-world interaction. We introduce a Scalable Online Post-training (SOP) system that enables online, distributed, multi-task post-training of generalist VLA models directly in the physical world. SOP tightly couples execution and learning through a closed-loop architecture in which a fleet of robots continuously streams on-policy experience and human intervention signals to a centralized cloud learner, and asynchronously receives updated policies. This design supports prompt on-policy correction, scales experience collection through parallel deployment, and preserves generality during adaptation. SOP is agnostic to the choice of post-training algorithm; we instantiate it with both interactive imitation learning (HG-DAgger) and reinforcement learning (RECAP). Across a range of real-world manipulation tasks including cloth folding, box assembly, and grocery restocking, we show that SOP substantially improves the performance of large pretrained VLA models while maintaining a single shared policy across tasks. Effective post-training can be achieved within hours of real-world interaction, and performance scales near-linearly with the number of robots in the fleet. These results suggest that tightly coupling online learning with fleet-scale deployment is instrumental to enabling efficient, reliable, and scalable post-training of generalist robot policies in the physical world.