level: research
stochastic-gradient langevin algorithms often use tamed denominators to handle drifts that are not globally lipschitz. when the denominator depends on the same stochastic gradient sample as the numerator, the taming step alters the stochastic oracle. this can introduce a stationary bias even if the original gradient estimate is unbiased. the paper shows that this bias arises because the denominator and numerator share noise, changing the effective target distribution.
the authors propose a structure-preserving method that fixes the denominator before sampling the oracle noise. they use localized deterministic envelopes to avoid unnecessary taming in typical regions. these envelopes are computed from the current parameter value without using the stochastic gradient. the approach keeps the stabilizing effect of taming while removing the bias from gradient-dependent denominators. the resulting kernels preserve the correct stationary distribution.
the theory decomposes the stationary error into two parts: bias from oracle-dependent taming and error from deterministic stabilization. within the deterministic-envelope family, the analysis provides error bounds and shows how to choose envelopes that balance stability and accuracy. experiments on benchmark models confirm that the method reduces bias compared to standard tamed sgmcmc. the framework is compatible with various taming schemes and can be applied to improve posterior sampling in bayesian inference.
why it matters: practitioners using tamed stochastic gradient mcmc can now avoid silent bias that corrupts posterior estimates, leading to more reliable bayesian inference in machine learning models.