标题： RealisMotion：世界空间中分解的人体运动控制与视频生成 显示英文标题

标题： RealisMotion: Decomposed Human Motion Control and Video Generation in the World Space

摘要： 生成具有真实且可控制动作的人类视频是一项具有挑战性的任务。 虽然现有方法可以生成视觉上引人注目的视频，但它们缺乏对四个关键视频元素的独立控制：前景主体、背景视频、人类轨迹和动作模式。 在本文中，我们提出了一种分解的人类动作控制和视频生成框架，该框架显式地将动作与外观、主体与背景以及动作与轨迹解耦，从而实现这些元素的灵活混合组合。 具体来说，我们首先构建一个基于地面的三维世界坐标系，并在三维空间中直接进行动作编辑。 通过焦距校准和坐标变换，将编辑后的二维轨迹反投影到三维空间中，随后进行速度对齐和方向调整；动作则通过动作库提供或通过文本到动作的方法生成。 然后，基于现代的文本到视频扩散变压器模型，我们将主体作为标记进行全注意力处理，沿通道维度拼接背景，并通过加法添加动作（轨迹和动作）控制信号。 这种设计使我们能够生成任何人任何地方的逼真视频。 在基准数据集和实际案例上的大量实验表明，我们的方法在元素级可控性和整体视频质量方面都达到了最先进的性能。 显示英文摘要

摘要： Generating human videos with realistic and controllable motions is a challenging task. While existing methods can generate visually compelling videos, they lack separate control over four key video elements: foreground subject, background video, human trajectory and action patterns. In this paper, we propose a decomposed human motion control and video generation framework that explicitly decouples motion from appearance, subject from background, and action from trajectory, enabling flexible mix-and-match composition of these elements. Concretely, we first build a ground-aware 3D world coordinate system and perform motion editing directly in the 3D space. Trajectory control is implemented by unprojecting edited 2D trajectories into 3D with focal-length calibration and coordinate transformation, followed by speed alignment and orientation adjustment; actions are supplied by a motion bank or generated via text-to-motion methods. Then, based on modern text-to-video diffusion transformer models, we inject the subject as tokens for full attention, concatenate the background along the channel dimension, and add motion (trajectory and action) control signals by addition. Such a design opens up the possibility for us to generate realistic videos of anyone doing anything anywhere. Extensive experiments on benchmark datasets and real-world cases demonstrate that our method achieves state-of-the-art performance on both element-wise controllability and overall video quality.