标题： Can3Tok：场景级3D高斯的规范3D标记化和潜在建模 显示英文标题

标题： Can3Tok: Canonical 3D Tokenization and Latent Modeling of Scene-Level 3D Gaussians

摘要： 3D生成已经取得了显著进展，然而，它仍然主要停留在物体级别。由于缺乏能够在3D场景级别数据上进行扩展潜在表示学习的模型，前馈3D场景级别生成很少被探索。与在有界规范空间中使用良好标记的3D数据进行训练的物体级别生成模型不同，由3D高斯点云（3DGS）表示的3D场景是无界的，并且在不同场景之间表现出尺度不一致，这使得为生成目的进行统一的潜在表示学习极具挑战性。在本文中，我们引入了Can3Tok，这是第一个能够将大量高斯原始对象编码为低维潜在嵌入的3D场景级别变分自编码器（VAE），它能够有效捕捉输入的语义和空间信息。除了模型设计，我们还提出了一种通用的3D场景数据处理流程来解决尺度不一致问题。我们在最近的场景级别3D数据集DL3DV-10K上验证了我们的方法，结果发现只有Can3Tok能够成功泛化到新的3D场景，而与其他方法相比，在训练过程中甚至在几百个场景输入上都无法收敛，并且在推理过程中表现出零泛化能力。最后，我们展示了图像到3DGS和文本到3DGS生成作为我们的应用，以展示其促进下游生成任务的能力。 显示英文摘要

摘要： 3D generation has made significant progress, however, it still largely remains at the object-level. Feedforward 3D scene-level generation has been rarely explored due to the lack of models capable of scaling-up latent representation learning on 3D scene-level data. Unlike object-level generative models, which are trained on well-labeled 3D data in a bounded canonical space, scene-level generations with 3D scenes represented by 3D Gaussian Splatting (3DGS) are unbounded and exhibit scale inconsistency across different scenes, making unified latent representation learning for generative purposes extremely challenging. In this paper, we introduce Can3Tok, the first 3D scene-level variational autoencoder (VAE) capable of encoding a large number of Gaussian primitives into a low-dimensional latent embedding, which effectively captures both semantic and spatial information of the inputs. Beyond model design, we propose a general pipeline for 3D scene data processing to address scale inconsistency issue. We validate our method on the recent scene-level 3D dataset DL3DV-10K, where we found that only Can3Tok successfully generalizes to novel 3D scenes, while compared methods fail to converge on even a few hundred scene inputs during training and exhibit zero generalization ability during inference. Finally, we demonstrate image-to-3DGS and text-to-3DGS generation as our applications to demonstrate its ability to facilitate downstream generation tasks.