level: research
many math-focused agent systems use a reviewer role to catch and fix errors, assuming that precise feedback will improve final answers. researchers tested this on 4,181 omni-math problems with gpt-oss-120b models. they compared a planner-executor-reviewer pipeline against a broadcast-style peer discussion. on easy problems, both methods performed similarly. but on harder problems, broadcast discussion reached higher accuracy than the pipeline, despite the pipeline's reviewer being more precise.
the key finding is that reviewer precision alone does not explain performance. the pipeline reviewer had a precision of 0.861 versus 0.644 for broadcast, yet its useful critiques were less likely to change the next candidate answer. this means that even when the reviewer correctly identified errors, the system often failed to incorporate the feedback into a better solution. the result was lower reviewer-guided repair in the pipeline setup.
the study suggests that how critique is integrated matters more than how accurate it is. broadcast discussion, where agents share and debate ideas, led to more effective uptake of feedback. this challenges the common design choice of relying on a single precise reviewer. for ai systems that solve complex reasoning tasks, fostering collaborative discussion may be more beneficial than optimizing individual reviewer accuracy.
why it matters: for ai systems that solve math problems, simply having an accurate reviewer is not enough; the system must effectively use the feedback to improve answers, which has implications for designing multi-agent reasoning architectures.