level: research
conformal bayes merges bayesian posterior predictives with conformal calibration to create prediction sets that are statistically valid and geometrically efficient. this work examines conformal bayes under label shift, where the distribution of labels differs between training and target domains. two complementary methods are identified to recover nominal coverage in the target domain, both relying on importance-weighted conformal calibration but operating through distinct mechanisms.
the first method, post-hoc calibration, adjusts the posterior predictive toward the target domain without altering the parameter posterior. it corrects the conformal threshold using an importance-weighted quantile. the second method, in-training adaptation, directly tilts the parameter posterior to the target domain. this produces a corrected predictive, and the highest predictive density region from this fitted target predictive serves as the prediction set. efficiency in this case depends on the model and does not guarantee optimality.
the paper provides a unified view of these strategies, highlighting their theoretical underpinnings and practical trade-offs. post-hoc calibration is simpler and leaves the model unchanged, while in-training adaptation modifies the learning process itself. both aim to maintain valid coverage when label distributions shift, a common challenge in real-world deployments. the analysis clarifies when each approach is preferable based on model assumptions and computational constraints.
why it matters: label shift is common in real-world ai applications, and these methods ensure prediction sets remain reliable when class distributions change, improving model trustworthiness.