source: arxiv artificial intelligence: recursive self-evolving agents via held-out selection
level: research
large language model agents often improve without weight updates by evolving natural language artifacts like reflections or playbooks. these methods are usually tested on a single benchmark. this work compares them directly using a shared local backbone. the authors introduce rsea, a recursive self-evolving agent with a three-layer state: an imperative strategy, reusable skills, and a procedural playbook. rsea rewrites all layers from its own trajectories across generations. it only keeps a candidate if it does not regress on a held-out split, using a strict keep-better gate.
the study evaluates rsea against six baselines: react, reflexion, gepa, awm, ace, and dynamic cheatsheet. tests run on four benchmarks: alfworld, gaia, tau-bench, and webshop. results show that many published gains vanish under controlled conditions. simple baselines often match or beat more complex methods. rsea consistently improves over generations without overfitting, unlike other approaches that degrade on held-out data. the held-out selection is key to its stability.
the findings suggest that self-evolution can work when guarded by a validation split. without it, agents tend to overfit to their training environments. the three-layer state provides a structured way to capture and reuse knowledge. this approach avoids the brittleness seen in single-artifact methods. the work highlights the importance of rigorous evaluation across multiple benchmarks. it also shows that simple, well-tested baselines remain strong competitors.
why it matters: it shows that self-improving ai agents need held-out validation to avoid overfitting, which is crucial for building reliable autonomous systems.
source: arxiv artificial intelligence: recursive self-evolving agents via held-out selection