source: arxiv statistics ml: anytime-valid federated conformal rag for llm swarms
level: research
federated conformal rag (fc-rag) gives distribution-free coverage guarantees for swarms of weak language models under bandwidth limits, but only at a fixed time horizon. this work extends fc-rag to anytime-valid sequential coverage, meaning the guarantees hold at every possible stopping time, even when the system adapts by recalibrating, increasing per-node bandwidth, or refreshing distilled student models. the extension requires no extra assumptions beyond those of fixed-horizon fc-rag.
a naive sequential combination fails because fc-rag's marginal coverage bound makes the betting e-process a non-supermartingale when calibration draws are adverse, so ville's inequality cannot be used. the proposed anytime-fc-rag solves this by introducing a summable per-step calibration-deviation budget. this budget converts the marginal bound into a strict conditional bound on a calibration-good event, paired with a truncated betting e-process that is a nonnegative supermartingale on the entire probability space.
the method ensures valid sequential coverage under predictable adaptive control, making it suitable for real-world deployments where model swarms must be adjusted on the fly. the approach maintains the original fc-rag benefits of being distribution-free and bandwidth-efficient while adding robustness to dynamic changes. this advance is particularly relevant for edge computing and federated learning scenarios where communication constraints and model heterogeneity are common.
why it matters: it enables reliable uncertainty quantification in adaptive, distributed ai systems, reducing the risk of overconfident predictions when models or data change over time.
source: arxiv statistics ml: anytime-valid federated conformal rag for llm swarms