source: arxiv statistics ml: beyond global divergences: a local-mass perspective on bayesian inference
level: research
bayesian inference often relies on global measures like kl divergence to compare distributions. this paper looks at local mass, the probability in tiny neighborhoods around parameter values. global objectives do not directly capture how this local mass behaves. the authors introduce two tools: mass index and regularized extended kl divergence. mass index records how fast local mass decays, either polynomially or logarithmically. regularized extended kl works even when distributions have singular parts, focusing on specific sets.
mass indices show how bayesian updating shifts local mass. when the likelihood has a power-log form, the mass index changes in a predictable way. if the support depends on the parameter, or is smoothed, the local mass scale can change based on how much mass stays near the parameter value. these effects are not obvious from global divergences alone. the paper gives explicit formulas for these shifts, helping to understand local concentration or dispersion after seeing data.
using local regularized extended kl, the authors prove inequalities comparing small-ball masses under the two directions of kl divergence. these include absolute, relative, and directional bounds. the results provide a finer-grained view of distributional differences. this matters for model selection, prior sensitivity, and understanding posterior concentration. the local perspective can diagnose when global measures hide important behavior, like heavy tails or singularities, that affect inference quality.
why it matters: understanding local mass helps diagnose when bayesian models concentrate or spread probability in unexpected ways, improving model checking and prior choice.
source: arxiv statistics ml: beyond global divergences: a local-mass perspective on bayesian inference