level: research
three methods for training language models to reason—grpo, dr. grpo, and dapo—appear to be different tricks. they all modify a single quantity: the standard deviation of correctness marks from a prompt's sampled answers. when a model answers a problem many times, an automatic checker labels each answer right or wrong. the standard deviation of these marks measures disagreement: it is largest when answers split evenly between right and wrong, and zero when they all agree.
grpo divides by this standard deviation. dr. grpo drops the division. dapo discards groups where the standard deviation is zero. each method was introduced as its own fix, but the paper shows they are three settings of one dial. for right-or-wrong rewards, the disagreement directly determines the size of the training update. the choice of operation on the standard deviation changes how the model learns from varied answer sets.
the finding unifies these approaches under a single framework. it clarifies that the core mechanism is controlling the influence of answer disagreement on policy updates. this insight can guide practitioners in selecting or tuning methods for reasoning tasks. it also suggests that future improvements may come from better handling of this disagreement signal rather than inventing new operations.
why it matters: understanding that these methods are variations of one idea helps ai practitioners choose and tune training techniques for reasoning models more effectively.