source: Hugging Face Blog: Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox

level: technical

microsoft and hugging face released thinkingbox-training, a method that turns database grading into reinforcement learning rewards. they built thinkingbox to catch agents that claim completion without actually finishing. the system checks final database state against expected outcomes. they used these executable checks as rewards to fine-tune qwen3.8-27b with grpo for 50 steps. the training pool came from tasks outside the benchmark, selected by offline policy-relative difficulty. this approach improved the model's repeatable success on a held-out benchmark of 507 tasks.

after training, observed 20/20 tasks rose from 38 to 107, a 2.8x increase. pass@1 improved from 51.70% to 60.78%, while pass@20 slightly declined from 453 to 446 tasks. the fine-tuned model became more consistent but slightly less broad. cost per dependable task dropped from $24.36 to $8.00, a 67.2% reduction. the training used 187 tasks, 18 groups of 8 rollouts per update, on 24 h100s. zero-variance groups were replaced dapo-style. system errors were refilled rather than scored as failures.

the benchmark tasks were never used in training, making evaluation out-of-distribution. the selection process probed each candidate task against the base policy to estimate success probability. tasks that were always solved or always failed were excluded because they provide no learning signal. the shared interpretation code ensures training and evaluation measure the same agent. this work shows that executable terminal-state checks can serve as both grader and reward, enabling post-training for stateful agents without human labels.

why it matters: this method lets teams improve an agent's reliability on database tasks using automated rewards, cutting the cost per dependable task by two-thirds.


source: Hugging Face Blog: Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox