标题： E-VRAG：利用资源高效检索增强生成提升长视频理解 显示英文标题

标题： E-VRAG: Enhancing Long Video Understanding with Resource-Efficient Retrieval Augmented Generation

摘要： 视觉-语言模型（VLMs）通过利用跨模态推理能力，在视频理解方面取得了显著进展。 然而，它们的有效性受到有限的上下文窗口和处理数千帧长视频所需的高计算成本的限制。 检索增强生成（RAG）通过仅选择最相关的帧作为输入来解决这一挑战，从而降低计算负担。 然而，现有的视频RAG方法在检索效率和准确性之间难以平衡，尤其是在处理多样且复杂的视频内容时。 为了解决这些限制，我们提出了E-VRAG，这是一种新颖且高效的视频RAG框架，用于视频理解。 我们首先应用一种基于分层查询分解的帧预过滤方法，以消除不相关的帧，从而在数据层面降低计算成本。 然后，我们使用轻量级VLM进行帧评分，进一步在模型层面降低计算成本。 此外，我们提出了一种帧检索策略，利用帧间评分的全局统计分布，以缓解使用轻量级VLM可能导致的性能下降。 最后，我们为检索到的帧引入了一种多视图问题回答方案，增强了VLM从长视频上下文中提取和理解信息的能力。 在四个公开基准上的实验表明， E-VRAG相比基线方法实现了约70%的计算成本减少，并且准确性更高，而无需额外训练。 这些结果证明了E-VRAG在提高视频RAG任务的效率和准确性方面的有效性。 显示英文摘要

摘要： Vision-Language Models (VLMs) have enabled substantial progress in video understanding by leveraging cross-modal reasoning capabilities. However, their effectiveness is limited by the restricted context window and the high computational cost required to process long videos with thousands of frames. Retrieval-augmented generation (RAG) addresses this challenge by selecting only the most relevant frames as input, thereby reducing the computational burden. Nevertheless, existing video RAG methods struggle to balance retrieval efficiency and accuracy, particularly when handling diverse and complex video content. To address these limitations, we propose E-VRAG, a novel and efficient video RAG framework for video understanding. We first apply a frame pre-filtering method based on hierarchical query decomposition to eliminate irrelevant frames, reducing computational costs at the data level. We then employ a lightweight VLM for frame scoring, further reducing computational costs at the model level. Additionally, we propose a frame retrieval strategy that leverages the global statistical distribution of inter-frame scores to mitigate the potential performance degradation from using a lightweight VLM. Finally, we introduce a multi-view question answering scheme for the retrieved frames, enhancing the VLM's capability to extract and comprehend information from long video contexts. Experiments on four public benchmarks show that E-VRAG achieves about 70% reduction in computational cost and higher accuracy compared to baseline methods, all without additional training. These results demonstrate the effectiveness of E-VRAG in improving both efficiency and accuracy for video RAG tasks.