标题： 超越平均值：使用高斯点云和嵌入集合的开放词汇3D场景理解 显示英文标题

标题： Beyond Averages: Open-Vocabulary 3D Scene Understanding with Gaussian Splatting and Bag of Embeddings

摘要： 基于3D高斯点云（3DGS）的新型视角合成取得了显著进展，实现了实时逼真的渲染效果。然而，高斯点云固有的模糊性对3D场景理解提出了挑战，限制了其在增强现实/虚拟现实（AR/VR）和机器人技术中的广泛应用。尽管近期工作尝试通过2D基础模型蒸馏来学习语义，但它们继承了根本性的局限：alpha混合在对象之间平均语义，使得3D级别的理解成为不可能。我们提出了一种范式转变的替代方法，完全绕过了通过可微分渲染获取语义。我们的关键见解是利用预分解的对象级高斯分布，并通过多视角CLIP特征聚合来表示每个对象，创建全面的“嵌入包”，以整体描述对象。这使得：(1) 通过将文本查询与对象级（而非高斯级）嵌入进行比较，实现准确的开放词汇对象检索，以及(2) 无缝的任务适应：将对象ID传播到像素用于2D分割或传播到高斯分布用于3D提取。实验表明，我们的方法有效克服了3D开放词汇对象提取的挑战，同时在2D开放词汇分割方面保持与最先进性能相当，确保最小的妥协。 显示英文摘要

摘要： Novel view synthesis has seen significant advancements with 3D Gaussian Splatting (3DGS), enabling real-time photorealistic rendering. However, the inherent fuzziness of Gaussian Splatting presents challenges for 3D scene understanding, restricting its broader applications in AR/VR and robotics. While recent works attempt to learn semantics via 2D foundation model distillation, they inherit fundamental limitations: alpha blending averages semantics across objects, making 3D-level understanding impossible. We propose a paradigm-shifting alternative that bypasses differentiable rendering for semantics entirely. Our key insight is to leverage predecomposed object-level Gaussians and represent each object through multiview CLIP feature aggregation, creating comprehensive "bags of embeddings" that holistically describe objects. This allows: (1) accurate open-vocabulary object retrieval by comparing text queries to object-level (not Gaussian-level) embeddings, and (2) seamless task adaptation: propagating object IDs to pixels for 2D segmentation or to Gaussians for 3D extraction. Experiments demonstrate that our method effectively overcomes the challenges of 3D open-vocabulary object extraction while remaining comparable to state-of-the-art performance in 2D open-vocabulary segmentation, ensuring minimal compromise.