标题： 变分自编码器的零方差梯度 显示英文标题

标题： Zero-Variance Gradients for Variational Autoencoders

摘要： 训练深度生成模型，如变分自编码器（VAEs），通常受到需要通过潜在变量的随机采样进行梯度反向传播的阻碍，这一过程会固有地引入估计方差，这可能会减缓收敛并降低性能。 在本文中，我们提出了一种新的观点，绕过了这个问题，我们称之为静默梯度。 与其改进随机估计器，我们利用特定的解码器架构来解析地计算期望ELBO，从而得到一个零方差的梯度。 我们首先为这种方法提供了理论基础，并在具有线性解码器的受控环境中展示了其优于现有估计器的优势。 为了将我们的方法推广到使用复杂、富有表现力的解码器的实际应用中，我们引入了一种新的训练动态，该动态在早期阶段使用精确的零方差梯度来指导编码器的训练，随后逐渐过渡到标准的随机估计器。 我们的实验表明，这种技术在多个数据集上 consistently 提高了现有基线模型的性能，包括重参数化、Gumbel-Softmax 和 REINFORCE。 这项工作通过结合解析计算的稳定性与深度非线性架构的表现力，为训练生成模型开辟了一个新方向。 显示英文摘要

摘要： Training deep generative models like Variational Autoencoders (VAEs) is often hindered by the need to backpropagate gradients through the stochastic sampling of their latent variables, a process that inherently introduces estimation variance, which can slow convergence and degrade performance. In this paper, we propose a new perspective that sidesteps this problem, which we call Silent Gradients. Instead of improving stochastic estimators, we leverage specific decoder architectures to analytically compute the expected ELBO, yielding a gradient with zero variance. We first provide a theoretical foundation for this method and demonstrate its superiority over existing estimators in a controlled setting with a linear decoder. To generalize our approach for practical use with complex, expressive decoders, we introduce a novel training dynamic that uses the exact, zero-variance gradient to guide the early stages of encoder training before annealing to a standard stochastic estimator. Our experiments show that this technique consistently improves the performance of established baselines, including reparameterization, Gumbel-Softmax, and REINFORCE, across multiple datasets. This work opens a new direction for training generative models by combining the stability of analytical computation with the expressiveness of deep, nonlinear architecture.