标题： 一种用于条件采样的黑盒去偏框架 显示英文标题

标题： A Black-Box Debiasing Framework for Conditional Sampling

摘要： 条件采样是贝叶斯统计和生成建模中的基本任务。 考虑从后验分布$P_{X|Y=y^*}$中进行采样的问题，对于某些观测值$y^*$，其中似然函数$P_{Y|X}$是已知的，并且我们给出了$n$个独立同分布的样本$D=\{X_i\}_{i=1}^n$，这些样本是从一个未知先验分布$\pi_X$中抽取的。 假设$f(\hat{\pi}_{X^n})$是由算法（例如 当$\hat{\pi}_{X^n}$是训练数据的经验分布时，条件生成模型或贝叶斯规则)。虽然对训练数据的随机性进行平均$D$，我们有$\mathbb{E}_D\left(\hat{\pi}_{X^n}\right)= \pi_X$，但由于$f$的非线性，我们没有$\mathbb{E}_D\left\{f(\hat{\pi}_{X^n})\right\}= f(\pi_X)$，导致偏差。在本文中，我们提出了一种黑盒去偏方案，以提高这种简单插值方法的准确性。 对于任何整数$k$以及似然函数的有界性和$f$的光滑性，我们生成样本$\hat{X}^{(1)},\dots,\hat{X}^{(k)}$和权重$w_1,\dots,w_k$，使得$\sum_{i=1}^kw_iP_{\hat{X}^{(i)}}$是$f(\pi_X)$的$k$阶近似，其中生成过程将$f$视为一个黑箱。 我们的生成过程在对训练数据的随机性进行平均时实现了更高的准确性，而不会降低方差，这可以解释为在生成模型中提高了记忆能力而不损害泛化能力。 显示英文摘要

摘要： Conditional sampling is a fundamental task in Bayesian statistics and generative modeling. Consider the problem of sampling from the posterior distribution $P_{X|Y=y^*}$ for some observation $y^*$, where the likelihood $P_{Y|X}$ is known, and we are given $n$ i.i.d. samples $D=\{X_i\}_{i=1}^n$ drawn from an unknown prior distribution $\pi_X$. Suppose that $f(\hat{\pi}_{X^n})$ is the distribution of a posterior sample generated by an algorithm (e.g. a conditional generative model or the Bayes rule) when $\hat{\pi}_{X^n}$ is the empirical distribution of the training data. Although averaging over the randomness of the training data $D$, we have $\mathbb{E}_D\left(\hat{\pi}_{X^n}\right)= \pi_X$, we do not have $\mathbb{E}_D\left\{f(\hat{\pi}_{X^n})\right\}= f(\pi_X)$ due to the nonlinearity of $f$, leading to a bias. In this paper we propose a black-box debiasing scheme that improves the accuracy of such a naive plug-in approach. For any integer $k$ and under boundedness of the likelihood and smoothness of $f$, we generate samples $\hat{X}^{(1)},\dots,\hat{X}^{(k)}$ and weights $w_1,\dots,w_k$ such that $\sum_{i=1}^kw_iP_{\hat{X}^{(i)}}$ is a $k$-th order approximation of $f(\pi_X)$, where the generation process treats $f$ as a black-box. Our generation process achieves higher accuracy when averaged over the randomness of the training data, without degrading the variance, which can be interpreted as improving memorization without compromising generalization in generative models.