source: arxiv statistics ml: xmse-aware adaptive empirical bayes estimation
level: research
empirical bayes estimators can match the first-order risk of maximum likelihood but often perform worse at second order when the shrinkage kernel is misaligned. recent work on excess mean squared error (xmse) shows that kernel-based eb can be inferior to ml. this paper turns that diagnostic into a design principle by proposing an xmse-aware mixed estimator that interpolates between ml and eb shrinkage.
the fixed-weight xmse of the mixed estimator is a scalar quadratic, which gives a closed-form oracle mixing weight that is never worse than both ml and the base eb estimator at the xmse scale. a plug-in implementation uses finite-sample xmse approximations and is proved consistent, with a second-order oracle regret rate for an interior oracle weight. the regret bound also transfers to the fixed-weight risk curve at the selected weight, a thresholded boundary rule, and extensions.
the approach provides a practical way to combine the strengths of ml and eb without prior knowledge of the true parameter structure. by focusing on second-order risk, it addresses a known weakness of standard eb methods. the method is supported by theoretical guarantees and can be applied in settings where eb is commonly used, such as multiple testing and small-area estimation.
why it matters: it offers a reliable way to improve estimation accuracy in high-dimensional data analysis by automatically balancing two standard methods.
source: arxiv statistics ml: xmse-aware adaptive empirical bayes estimation