标题： REFA：虚拟现实中的实时第一视角面部动画 显示英文标题

标题： REFA: Real-time Egocentric Facial Animations for Virtual Reality

摘要： 我们提出了一种新颖的系统，用于使用从嵌入虚拟现实（VR）头显中的红外摄像头捕获的自我视角视图实时跟踪面部表情。 我们的技术使任何用户都能以非侵入性的方式准确驱动虚拟角色的面部表情，而无需进行耗时的校准步骤。 我们系统的核心是一种基于蒸馏的方法，在来自多个来源的异构数据和标签上训练机器学习模型，\eg 合成和真实图像。 作为我们数据集的一部分，我们使用轻量级采集设置（包括一部智能手机和一个带有额外摄像头的定制VR头显）收集了18k个不同的受试者。 为了处理这些数据，我们开发了一个强大的可微分渲染流程，使我们能够自动提取面部表情标签。 我们的系统为虚拟环境中的交流和表达开辟了新途径，应用于视频会议、游戏、娱乐和远程协作。 显示英文摘要

摘要： We present a novel system for real-time tracking of facial expressions using egocentric views captured from a set of infrared cameras embedded in a virtual reality (VR) headset. Our technology facilitates any user to accurately drive the facial expressions of virtual characters in a non-intrusive manner and without the need of a lengthy calibration step. At the core of our system is a distillation based approach to train a machine learning model on heterogeneous data and labels coming form multiple sources, \eg synthetic and real images. As part of our dataset, we collected 18k diverse subjects using a lightweight capture setup consisting of a mobile phone and a custom VR headset with extra cameras. To process this data, we developed a robust differentiable rendering pipeline enabling us to automatically extract facial expression labels. Our system opens up new avenues for communication and expression in virtual environments, with applications in video conferencing, gaming, entertainment, and remote collaboration.