source: arxiv artificial intelligence: what drives interactive improvement from feedback?
level: research
researchers tested when natural-language feedback leads to real improvement in language agents, beyond what repeated attempts alone can achieve. they used a student-teacher setup across four benchmarks: omni-math, codeforces, bbeh linguini, and arc-agi1. thirteen open-weight models acted as both students and teachers. the study compared external feedback, self-feedback, and unguided self-refinement while varying interaction history, task difficulty, and teacher access to privileged information.
the results show that multi-turn improvement often does not mean the agent is using feedback effectively. self-generated feedback added little beyond unguided self-refinement, where the model simply tries again without any feedback. in contrast, the strongest external teachers produced substantially larger gains that were specifically due to feedback. this suggests that apparent progress in multi-turn settings can be misleading, driven by resampling, format correction, or extra test-time computation rather than learning from feedback.
the findings highlight the need to separate feedback-specific effects from other sources of improvement when evaluating language agents. without careful controls, higher final accuracy may be wrongly attributed to feedback quality. the study provides a protocol for isolating the true impact of feedback, which is crucial for building agents that genuinely learn from interaction. the work also shows that current self-feedback methods are weak, pointing to the need for better teacher models or feedback mechanisms.
why it matters: for ai and data science, this study warns that apparent gains from multi-turn interaction may be fake, helping practitioners design better evaluation and feedback systems.
source: arxiv artificial intelligence: what drives interactive improvement from feedback?