level: research
conformal prediction is used in drug discovery to give reliable error estimates. it promises that prediction sets contain the true label with at least a set probability. but this guarantee breaks on imbalanced datasets. across four datasets, marginal conformal prediction met its global 90% coverage target while minority class coverage dropped to 64.8% on blood-brain-barrier penetration and just 4.2% on clinical-trial toxicity. the rare class was almost entirely missed.
the problem is not tied to a specific model. a random forest, a graph network, and a frozen chemical language model all showed the same failure. in every case, the minority class was under-covered with high statistical significance. the severity depended on how well the model was calibrated on rare labels, not on the model architecture. a conservation identity explains this: the minority's coverage shortfall equals the majority's surplus multiplied by the class imbalance ratio.
a class-conditional conformal prediction method fixes the issue. it applies the conformal procedure separately to each class. this restores the promised coverage for both majority and minority classes. the approach is simple and does not require changing the underlying model. it ensures that rare but critical outcomes, like toxicity, are not overlooked during virtual screening.
why it matters: in drug discovery, missing a rare toxic compound can be disastrous; this fix ensures conformal prediction works reliably for imbalanced safety-critical data.