标题： 基于视觉语言模型的开放式分层流式视频理解 显示英文标题

标题： Open-ended Hierarchical Streaming Video Understanding with Vision Language Models

摘要： 我们引入了分层流视频理解，这是一个结合在线时间动作定位与自由格式描述生成的任务。 由于具有分层和细粒度时间标注的数据集稀缺，我们证明了大型语言模型可以有效地将原子动作分组为更高级的事件，从而丰富现有的数据集。 然后我们提出了OpenHOUSE（开放式的分层在线事件理解系统），它将流动作感知扩展到动作分类之外。 OpenHOUSE具有一种专门的流模块，能够准确检测紧密相邻动作之间的边界，性能几乎达到现有方法直接扩展的两倍。 我们设想流动作感知的未来在于强大生成模型的整合，而OpenHOUSE代表了这一方向的关键一步。 显示英文摘要

摘要： We introduce Hierarchical Streaming Video Understanding, a task that combines online temporal action localization with free-form description generation. Given the scarcity of datasets with hierarchical and fine-grained temporal annotations, we demonstrate that LLMs can effectively group atomic actions into higher-level events, enriching existing datasets. We then propose OpenHOUSE (Open-ended Hierarchical Online Understanding System for Events), which extends streaming action perception beyond action classification. OpenHOUSE features a specialized streaming module that accurately detects boundaries between closely adjacent actions, nearly doubling the performance of direct extensions of existing methods. We envision the future of streaming action perception in the integration of powerful generative models, with OpenHOUSE representing a key step in that direction.