source: arxiv statistics ml: toward simultaneously optimal regret in u-calibration
level: research
u-calibration studies online forecasting where predictions must work for any downstream agent, guaranteeing low regret for all proper loss functions. existing methods achieve the best possible worst-case regret of order square root t for every bounded proper loss. however, they cannot adapt to easier losses. for smooth losses like squared loss, they still suffer square root t regret, while the optimal is logarithmic in t. this gap means current algorithms are inefficient when losses are nice.
the new work shows this limitation is not fundamental. the authors design a single forecasting algorithm that simultaneously gets near square root t regret for every bounded proper loss and logarithmic regret for every bounded smooth proper loss. more generally, it also achieves logarithmic regret for losses smooth relative to the log-barrier, covering some non-lipschitz cases. the method uses a novel variant of the follow-the-regularized-leader framework with a carefully chosen regularizer.
the algorithm works by maintaining a distribution over experts and updating it based on past losses. it uses a time-varying learning rate and a special regularizer that balances between the general and smooth loss cases. the analysis shows that for smooth losses, the regret bound improves from square root t to log t, matching the best possible. experiments on synthetic data confirm the theoretical findings, showing faster convergence for smooth losses compared to prior u-calibration methods.
why it matters: this algorithm makes online predictions more efficient for common smooth losses like squared error, reducing regret from square root to logarithmic, which is crucial for applications needing fast adaptation.
source: arxiv statistics ml: toward simultaneously optimal regret in u-calibration