level: technical
large language models often produce chain-of-thought reasoning that looks logical but may not actually rely on the given premises. researchers introduced interventional grounding audits, a black-box test that works at the step level. the method intervenes on a single premise by replacing its target predicate with a new symbol, then reruns the model to see if each reasoning step's normalized conclusion changes. this checks whether the step genuinely depends on that premise.
the approach was tested on prontoqa, a synthetic multi-hop deductive reasoning benchmark with known proof trees and step-level premise dependencies. using gpt-4o on 50 problems, the audit achieved an f1 score of 0.806 for detecting proof-tree dependencies, and 0.885 for predicate-determining dependencies, with 100% recall. this significantly outperformed a self-consistency baseline, which scored only 0.343, with non-overlapping 95% bootstrap confidence intervals.
the study also found that 66% of correctly solved problems contained reasoning steps that did not depend on the premises, suggesting that models often produce plausible-sounding but ungrounded chains of thought. the method provides a way to audit model reasoning without access to internal weights, making it useful for evaluating deployed systems. it highlights the gap between apparent and actual logical grounding in llm outputs.
why it matters: it offers a practical tool to verify if llm reasoning is truly grounded in facts, helping to catch plausible but incorrect logic in ai systems.