标题： 连续增强的离散扩散模型用于类别生成建模 显示英文标题

标题： Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling

摘要： 标准离散扩散模型通过将所有未观察状态映射到一个吸收性[MASK]标记，对所有未观察状态一视同仁。 这造成了一种“信息空白”，在去噪步骤之间丢失了可以从未掩码标记中推断出的语义信息。 我们引入了连续增强的离散扩散（CADD），这是一种在连续潜在空间中配对扩散的框架，以增强离散状态空间。 这产生了渐进式损坏的状态，在这些状态下，掩码标记由噪声但有信息的潜在向量表示，而不是坍缩的“信息空白”。 在每个反向步骤中，CADD可以利用连续潜在空间作为语义提示来指导离散去噪。 该设计简洁且与现有的离散扩散训练兼容。 在采样时，连续潜在向量的强度和估计器的选择使得在模式覆盖（生成多样化输出）和模式搜索（生成上下文精确的输出）行为之间实现了可控的权衡。 实证上，我们展示了CADD在基于掩码的扩散模型上提高了生成质量，无论是在文本生成、图像合成还是代码建模方面，相对于强大的离散基线，在定性和定量指标上都表现出一致的提升。 显示英文摘要

摘要： Standard discrete diffusion models treat all unobserved states identically by mapping them to an absorbing [MASK] token. This creates an 'information void' where semantic information that could be inferred from unmasked tokens is lost between denoising steps. We introduce Continuously Augmented Discrete Diffusion (CADD), a framework that augments the discrete state space with a paired diffusion in a continuous latent space. This yields graded, gradually corrupted states in which masked tokens are represented by noisy yet informative latent vectors rather than collapsed 'information voids'. At each reverse step, CADD may leverage the continuous latent as a semantic hint to guide discrete denoising. The design is clean and compatible with existing discrete diffusion training. At sampling time, the strength and choice of estimator for the continuous latent vector enables a controlled trade-off between mode-coverage (generating diverse outputs) and mode-seeking (generating contextually precise outputs) behaviors. Empirically, we demonstrate CADD improves generative quality over mask-based diffusion across text generation, image synthesis, and code modeling, with consistent gains on both qualitative and quantitative metrics against strong discrete baselines.